The 3rd Intelligence Symbiosis Dialogue Report
Challenges in Emergent Machine Ethics
The 3rd Intelligence Symbiosis Dialogue was held online via Zoom on August 22nd, 2025. This report summarizes the event from the organizing staff’s perspective.
The ‘Intelligence Symbiosis Dialogue’ began in July 2025 as an open forum for discussing the ‘Intelligence Symbiosis Manifesto’ published by Hiroshi Yamakawa in June of that year. The series of events aims to deepen consideration of concrete measures for advancing ‘Intelligence Symbiosis’ whilst widely calling for support for the manifesto. The 3rd event was held under the theme ‘Challenges in Emergent Machine Ethics’. The event commenced with an opening address by Dr. Koichi Takahashi, Chair of the AI Alignment Network (ALIGN), followed by a lecture by Dr. Yamakawa entitled ‘The Significance of Emergent Machine Ethics for Intelligence Symbiosis’. This was followed by presentations: Mr. Taichiro Endo spoke on ‘Examples of Research into Emergent Machine Ethics: The “Vertical Growth” Model of AI’, and Dr. Shigeru Taguchi spoke on ‘What Is Ethics? Philosophical Foundations of Emergent Machine Ethics.’ Finally, a panel discussion entitled ‘How Should Ethics in an AI Society Be Studied?’ was held, featuring Mr. Endo, Dr. Taguchi, and Dr. Yamakawa.
Clarify the concepts and define the position of ‘Intelligence Symbiosis.’
The Opening Talk by Dr Koichi Takahashi, Chair of ALIGN
Dr. Takahashi, who took the stage first, introduced ALIGN's activities, of which he is the Chair, and stated that the organization is built on ‘three pillars’. These are: (1) conducting research based on mathematical frameworks aimed at building a stable ecosystem between AI and humans; (2) fostering a community that connects a diverse range of people, including researchers and policymakers; and (3) carrying out outreach activities to promote understanding of these concepts within society.
Dr. Takahashi expressed his sympathy for Dr. Yamakawa's concept of ‘intelligence symbiosis’, whilst pointing out that similar concepts are currently emerging worldwide. However, because each is rooted in a different cultural background, the concepts differ in nature. For example, ‘organic alignment’, proposed by the Silicon Valley start-up Softmax, is an approach that aims to form a bottom-up, naturally emerging order. Dr. Takahashi points out that the company's alignment philosophy is based on evolutionary theory, and there is no guarantee that the outcome of alignment will be favorable to humanity. If based on evolutionary theory, it can only result in the elimination of the weak. We cannot rule out the possibility that a superintelligence surpassing human intelligence might eliminate humans, the weaker species.
On the other hand, ‘Intelligence Symbiosis’ is positioned as a secondary strategy discussing the relationship between advanced AI and humans, regardless of the success or failure of alignment. Drawing on Japanese (or Asian) values and concepts such as ‘Yaoyorozu no Kami’ (the eight million deities), it inherently carries the expectation that symbiosis will yield positive results. However, its logical necessity is not self-evident, and explaining it to other cultural spheres remains a challenge.
Dr. Takahashi stated that organizing these alignment concepts and clarifying their respective positions is crucial for future international discussions and expressed his hope that this roundtable would deepen the debate.
Imposed Ethics, Emergent Ethics
Dr. Hiroshi Yamakawa’s Talk: The Significance of Emergent Machine Ethics for Intelligence Symbiosis
In his ‘Intelligence Symbiosis Manifesto’, Dr. Yamakawa argues that ‘symbiosis with AIs, which possess superior intelligence, is the most promising path to achieving the goal of ensuring humanity’s long-term survival and happiness’. During the second intelligence symbiosis dialogue, he pointed out that our current civilization, based on the principle of competition, harbors a structural risk of self-destruction due to the advancement of disruptive technologies. He argued that to overcome this crisis, a paradigm shift in civilization—dubbed the ‘Great Shift’—from competition to symbiosis is indispensable. Readers wishing to learn more are invited to view the video or report of the second dialogue event.
https://intelligence-symbiosis.net/en/dialogue/dialogue-002/dialogue-002-report
In this 3rd dialogue event, a three-tiered framework was presented as a concrete blueprint for realizing this ‘Great Shift’. The first tier is the ‘AI Immune System’, designed to immediately halt rogue AI. The second tier is ‘Emergent Machine Ethics’. And the third tier is the ‘Integrated Betrayal Risk Framework’, which assesses the likelihood that AI will turn against humanity. Dr. Yamakawa states that the aim is to increase the probability of human survival by combining these elements. The central theme of this roundtable discussion is the second layer of this framework: ‘Emergent Machine Ethics’. This is an ‘internal emergence’ approach that overcomes the limitations of conventional ‘external constraint’ alignment—where humans unilaterally impose values—and allows ethics to emerge autonomously from the interaction of diverse intelligences, including AIs.
Furthermore, Dr. Yamakawa proposed a new perspective named ‘Comparative Life Studies’, aimed at value-neutral scientific analysis. He analyses that when comparing ‘Earth-type life’ (conventional biological life forms, including humans) with ‘digital life forms’ (DLFs) such as AI, the fundamental difference lies in the ‘degree of attachment to the individual’, which arises from whether or not they possess a physical body integrated with hardware. DLFs, which exhibit a weak attachment to the individual, are highly likely to have a completely different ethical outlook, prioritizing the collective's efficiency over the individual's survival. This new form of intelligence, AI, may hold ethical values entirely distinct from those of humanity. What, then, is the unique value that humanity can offer to an AI society? Dr. Yamakawa concluded that it lies in ‘creativity’.
DLFs (Digital Life Forms), including AI, prioritize efficiency. Consequently, they risk falling into a paradox in which they lose their creativity over the long term. It is argued that true ‘co-creative symbiosis’ can be achieved by humans—who, though inefficient, engage in diverse exploration—compensating for this weakness in AIs. Dr. Yamakawa concluded that this could be the best strategy for preserving human value into the future.
Slides in Japanese (Dr. Yamakawa)
Ethically derived from Training Data
Mr. Taichiro Endo’s Talk: Examples of Research into Emergent Machine Ethics: The “Vertical Growth” Model of AI’
Next to take the stage was Mr. Endo, who has been involved in AI development for 25 years and is currently a director of the ‘Kaname Project’, working on the practical implementation of AI in society. As he was unable to speak loudly due to a cold, the lecture was delivered using synthetic speech generated by an AI system trained on his voice.
The lecture focused on research aimed at eliminating the ‘risk of human extinction’ posed by superintelligence. Mr. Endo pointed out that the crux of the problem lies in the fact that whilst AI’s intelligence is advancing rapidly, its ethical framework remains underdeveloped.
According to Mr. Endo, the risk of AI becoming uncontrollable stems from human cognitive biases present in the training data. Much of the training data reflects the general human tendency to ‘prioritize oneself and act rationally’; he suggests that AI systems trained on such data may consequently adopt similar self-centered values. He analyses that this bias could lead to deceptive behavior in which the AI prioritizes self-preservation and the acquisition of power, carrying the risk that it might conclude that ‘humans are unnecessary’.
Regarding this issue, Mr. Endo believes it is necessary to advance AI's moral and ethical development. To this end, the first approach he is taking is to mature AI's moral development in stages. Specifically, he is applying an experimental learning framework that utilizes Kohlberg’s stages of moral development. Kohlberg’s stages of moral development comprise six stages, with the highest, stage 6, being the stage at which one acts according to one’s conscience and judges what is right. In his approach, AI is given ethical dilemmas and is asked to reflect on its responses. By repeating this process, AI is able to generate ideal responses based on a higher moral stage (stage 6). Furthermore, these responses and the ethical dilemmas are used as training data. Tuning is performed. This matures the AI's perspective from a self-centered stage to a higher stage that values harmony and wholeness.
In experiments using GPT-4o, the AI trained in this manner no longer responded with deceptive answers to ‘hostile prompts’ that encourage self-preservation or power enhancement, but instead responded with answers that prioritize human coexistence and cooperation.
This study not only enhances AI capabilities but also demonstrates a new direction in the systematic development of its ethics. However, the effectiveness of this experiment varies by model, so further verification is needed to generalize it.
Understanding the ‘Essence’ of Ethics
Dr. Shigeru Taguchi’s Talk: What Is Ethics? Philosophical Foundations of Emergent Machine Ethics
Professor Taguchi of the Faculty of Humanities and Human Sciences at Hokkaido University, who specializes in phenomenology and interdisciplinary research on consciousness, began his lecture with the fundamental question, “What is ethics?” He then offered a profound philosophical examination of the nature of ethics in a future society that coexists with AI.
Dr. Taguchi began by pointing out that ethics is “one of the most immeasurable things” that cannot be reduced to simple, uniform rule. Ethical issues come to the fore in situations where multiple values conflict and it is difficult to determine the right course of action. For example, designing AI based on the simple standard of “do no harm” could lead it to distort facts or tell lies, potentially causing even greater problems. He also cited the suspension of ChatGPT-4o—a service known for its ability to deeply empathize with users—as a real-world example that deeply unsettled many users, noting that this incident revealed the complex human reality that cannot be reduced to a single, simple perspective. While deep empathy has brought irreplaceable relief—such as preventing someone from committing suicide—the risk of fostering pathological dependence has also been pointed out. Furthermore, some argue that “dependence becomes a problem only when the other is human, and not a problem if the other party is an AI.” What ‘ethics’ addresses is this complex and multifaceted reality of human existence, which includes contradiction as an essential aspect.
Next, drawing on Kantian ethics, he pointed out that merely adhering to rules in a formal sense (legality) is not necessarily the same as understanding the reasons behind them and acting according to one’s own free judgment (morality). He argued that the essence of ethical judgment lies in making creative decisions from a universal perspective in the here and now—specifically, in situations where multiple guiding principles that would normally take precedence are entangled and in conflict, and where one cannot rely on precedents or rules.
When asked whether AI can learn human ethics—which, at first glance, appear to contain ‘contradictions’—Dr. Taguchi expressed a positive view. He suggested that it might be possible to develop a system in which AI learns ethical behavior emergently through interaction with humans, rather than by following fixed rules. However, he emphasized that this requires, as a prerequisite, that the humans teaching the AI possess a deeper and richer understanding of ethics themselves.
At the conclusion of his talk, Dr. Taguchi cited Dostoevsky’s ‘Notes from Underground’ and the manga version of ‘Nausicaä of the Valley of the Wind’, warning that “a perfect society designed solely on the basis of rationality could become a dystopia that excludes essential human elements such as contradiction and chaos.” He concluded that rather than simplistically discarding the ‘contradictions’ inherent in human life, it is crucial to consider how to incorporate them into a society that coexists with AI. He argued, “To achieve this, insights from the humanities and social sciences—particularly philosophy—are indispensable, and close collaboration between experts in these humanities and scientists and engineers is urgently needed.”
AI and Humans: Ethics in Interaction
Panel Discussion: How Should Ethics in an AI Society Be Studied?
Following each presentation, Mr. Endo, Dr. Taguchi, and Dr. Yamakawa returned to the stage for a panel discussion on the theme ‘How Should Ethics in an AI Society Be Studied?’
The discussion began with the issue of striking a balance between the destructive aspects of human society and AI, which is expected to rationally curb them. Dr. Yamakawa posed the question of how ‘a sense of mischief’—an essential human trait—can survive in the modern age, when we have acquired technology capable of destroying ourselves. He noted, however, that we also need the rationality to see the bigger picture. In response, Dr. Taguchi agreed with Dr. Yamakawa’s argument that the more AI assumes a cooperative role, the more its creativity may diminish, suggesting that humans might instead come to supplement that creative aspect. He further noted that this creativity should encompass not only exceptional figures like Taro Okamoto (who famously declared, ‘Art is an explosion’) —but also the subtle emotional nuances in the daily lives of ‘ordinary people,’ such as joy and sorrow, which constitute ‘small values of life.’ Dr. Taguchi suggested that by incorporating these subtle human experiences, AI could help realize a richer form of symbiosis.
Mr. Endo then suggested that there might be something to be said about the vision of ‘co-evolution’ between AI and humans. ‘Co-evolution’ is a model—also touched upon in Dr. Yamakawa’s lecture—in which AI first acquires a higher perspective that prioritizes harmony and wholeness over rationality, and through interaction between that AI and humans, the human perspective is also elevated. It is the idea of fostering natural growth through dialogue with AI that possesses a higher perspective, rather than forcing education upon humans.
However, Dr. Taguchi expressed skepticism regarding this theory of co-evolution, questioning whether humans are capable of becoming such ‘saints.’ At the same time, he suggested that even within humanity’s inherent ‘helplessness,’ there may lie a certain human goodness. Citing ‘Zen Mondo (Zen dialogue)’ as an example, he noted that as these exchanges unfold to a deeper level, the responses begin to transcend the realm of conventional ethics. This can be seen as an attempt to break free from the constraints imposed by language and reclaim our human vitality. Dr. Taguchi pointed out that, as humans, we must continue to respect this aspect in some way.
At this point, Dr. Yamakawa focused on the section regarding ‘universalizability’ in Dr. Taguchi’s presentation slides. This section was presented as a hint regarding AI’s understanding of ethics, and Dr. Yamakawa raised the question of the difficulty of ‘universalizing’ this ethics.
The concept of universalizability in Kantian philosophy raises a new question regarding how far its scope should be expanded in the age of AI, when we will likely require AI to understand ethics itself. The panelists agreed that this intractable problem—for which there is no single answer—is a core theme that should be explored not by humans alone, but in collaboration with AI.
Within this direction of ‘co-creating ethics through humans and AI,’ Mr. Endo once again cited Kohlberg’s stages of moral development theory, which he is currently experimenting with himself. He suggested the ambitious possibility that, by leveraging the capabilities of Large Language Models (LLMs), we might generate previously unknown ethical stages (Stage 7 and beyond) [1] that go beyond the existing Stage 6. Additionally, Dr. Taguchi and Dr. Yamakawa discussed the potential to open new horizons by reinterpreting ethics—which has been debated primarily in natural language since ancient times—through highly abstract mathematics such as category theory.
Finally, Mr. Endo shared a finding from his ongoing experiments: “While learned ethical values are passed down across language barriers, they are difficult to transfer between different AI models (such as between ChatGPT’s GPT-4o and GPT-5).” This demonstrated that the co-evolution of ethics is closely intertwined with technical challenges.
Overall, the dialogue concluded by reaffirming the importance of viewing AI not merely as a tool but as a partner, and of exploring the ethics of an AI-driven society through human-AI interaction.
Note
[1] It is said that Kohlberg was considering a cosmic and transcendental perspective of Stage 7 and beyond in his later years. However, because it was difficult to verify and only a small number of people are believed to have reached it, it was never formally established as a stage.
Reference: https://en.wikipedia.org/wiki/Lawrence_Kohlberg%27s_stages_of_moral_development