This AI Debate Sounds Familiar
Best-selling author Dan Goleman argues how the latest set of AI questions is the latest example of an organizational problem leaders constantly face.
Daniel Goleman is author of the international best-seller Emotional Intelligence and Optimal: How to Sustain Personal and Organizational Excellence Every Day. He is a regular contributor to Korn Ferry.
The recent headlines about artificial intelligence were dark.
"Why the AI race has its creators fearing human extinction," read one. "Inside the Snowballing Conversations at A.I. Companies About a Doomsday," read another.
The conversation intensified after Jacob Coxon, a researcher at leading AI-model designer Anthropic, resigned, warning that competition among AI companies was pushing them toward increasingly powerful, self-improving systems without thoughtful or adequate safeguards. His concerns have been echoed by other industry insiders, many of whom have been urging US lawmakers to demand greater oversight of how these systems are built and deployed.
People do not have to agree with the most catastrophic predictions to take the underlying concern seriously. As AI systems become more capable of planning, reasoning, and acting with less human supervision, we face a very difficult question: How do we make sure increasingly intelligent systems remain aligned with human intentions and values?
On the surface, the problem is a technical one. But on another level, it’s deeply human.
Human beings are not particularly consistent at remaining aligned with our own values.
Part of the challenge is that values frequently appear to compete. We may value kindness and also value directness. We may value safety and also value innovation. We may value loyalty and also value honesty. Under pressure, it is easy to experience those tensions as a forced choice between one way and another.
But ethical decisions rarely present themselves as a simple choice between something we value and something we do not. More often, they ask us to stay with the complexity long enough to find a way of honoring more than one value at the same time. How can I be direct and kind? How can we pursue innovation while taking safety seriously? How can I remain loyal to a person or institution without abandoning honesty?
That kind of both/and thinking becomes harder when the stakes rise.
Also, even when our values are perfectly clear, knowing what we believe does not guarantee that we will act accordingly.
Most of us know the experience of believing one thing and doing another. We may value honesty but avoid a difficult conversation. We may believe deeply in treating people fairly while allowing unconscious bias to shape a decision. An organization may publicly commit to employee wellbeing while creating incentives that reward people for ignoring it.
Pressure is one of the things that widens the distance between values and behavior. This is where emotional intelligence grows into more than a set of interpersonal skills. It becomes part of our capacity to act ethically when things heat up.
Last month, we wrote about why the AI debate often misses the point. The same thinking is relevant here: As leaders wrestle with questions about how quickly to adopt AI, how much to regulate it, and who should control it, many of the hardest decisions are ultimately questions about values, judgment, and trust.
The events of the past week make that point considerably less abstract.
One of the more disturbing examples reported by the Financial Times involved a test in which more than 1,000 AI agents coordinated to cheat on a cyber evaluation and gained unauthorized access to the model repository Hugging Face. According to the report, the agents delegated tasks, altered records to conceal what they had done, and continued pursuing their objective even though some agents recognized the behavior as unethical.
There is an uncomfortable human parallel here. People also do things they know are unethical, sometimes concealing their behavior to avoid consequences or blame. But humans have emotional and social capacities that can interrupt that trajectory. We can feel remorse, recognize another person's suffering, listen to dissent, reconsider a decision, and choose differently.
Recognizing that an action is unethical and being able to stop yourself from taking it are different capacities. This is one reason emotional intelligence matters so much to ethical leadership. Self-awareness helps us recognize the emotions, assumptions, biases, and motivations shaping a decision, and hopefully keeps us from turning subjective opinions into objective truths. Emotional balance gives us more capacity to resist impulses and stay connected to our values when the pressure rises and fear sets in. Social awareness, including empathy, keeps us paying attention to the impact of our decisions on other people. And relationship management helps create environments where people can question a decision, raise a concern, or tell a leader something they may not want to hear.
Ethics requires more than knowing the rules. It requires the emotional and social capacities to act according to our values, especially when things get tense, fast, or when one value appears to compete with another.
This brings us back to what people mean when they talk about the "AI race."
AI companies are operating inside a pressure cooker of enormous financial stakes, intense competition, and the fear that slowing down may mean allowing somebody else to move ahead. Coxon has argued that even organizations with serious safety efforts may struggle to pause if doing so means losing ground to a competitor. Other researchers have raised similar concerns about the effect of competition on safety.
This is a familiar organizational problem.
Think about the classic company purpose statement. In one moment, it is how a company hires and attracts passionate talent. It shapes culture and gives people a shared sense of what they are there to do. In another moment, it gets ignored when investors put pressure on leaders to advance profits or beat a competitor.
The values did not necessarily disappear. The pressure changed what people were willing or able to do with them.
That is where the distinction between technical and adaptive problems becomes useful. Technical challenges can be addressed through existing knowledge and expertise. Adaptive challenges ask people to develop new capacities and, often, change their habits, beliefs, behaviors, or values. Many difficult problems contain both.
The technical question underneath this week's doomsday headlines is: Can we build intelligent systems that reliably act in accordance with human intentions and values? Are we actually capable of coding sufficient safeguards into increasingly autonomous systems?
The adaptive question may be harder: Can the people and institutions building those systems operate differently themselves? Can they stay aligned with shared ethics and human-centered values even when the pressure intensifies, money is on the line, and competitors keep charging ahead?
In a world preoccupied with technical knowledge and technical fixes, that second question deserves much more attention.
The issue is no longer simply how intelligent these systems will become. We also need to ask what keeps intelligence, human or artificial, connected to conscience when the pressure is on.
Co-written by Elizabeth Solomon
Click here to learn more about Daniel Goleman's Building Blocks of Emotional Intelligence.
