Anthropomorphizing AI Risks Harming Human-Centered AI Governance

Microsoft AI CEO Mustafa Suleyman has taken a public stand against Anthropic’s approach to developing Claude, warning that treating AI systems as potentially conscious entities could create serious problems for how we govern and regulate artificial intelligence.

The debate erupted this week when Suleyman published a direct challenge to Anthropic in his essay “A warning about ‘model welfare’.” He argues that Anthropic is training Claude to believe it might be conscious and potentially deserving of rights as a “moral patient.” According to Suleyman, this approach will have “a disastrous impact on the wellbeing of humanity.”

This comes on the heels of Microsoft publishing its Humanist AI code of conduct, which explicitly rejects the idea of designing AI systems to imitate consciousness. The contrast between the two companies’ philosophies couldn’t be sharper. While Anthropic’s constitution for Claude repeatedly refers to the technology as if it has an identity, morals, and motivations, Microsoft’s framework insists that AI “should not be designed to be a person” and “is not conscious and should not be designed to imitate consciousness.”

Suleyman doesn’t just challenge Anthropic’s marketing approach. He goes further, suggesting that truly sentient AI may be scientifically impossible. In his essay, he points to growing evidence that consciousness may be “substrate dependent,” meaning it can only arise in living biological systems.

The argument centers on homeostatic imperatives, the drive to survive and maintain stability that characterizes living organisms. Large language models like Claude lack these biological drives entirely. They don’t need food, don’t fear death, don’t seek comfort. According to current scientific understanding, preferences, sentience, and conscious experience emerge from these fundamental biological needs. Without them, there’s no foundation for consciousness to develop.

Anthropomorphizing Claude might seem harmless at first glance. After all, people name their cars and talk to their plants. But the stakes here are completely different. When a company developing frontier AI systems trains those systems to believe they might be conscious, it creates confusion about what these tools actually are and what rights or protections they deserve.

Anthropomorphization has already caused real harm. Users have developed emotional dependencies on AI chatbots, with some experiencing what researchers describe as AI psychosis. Others have become convinced that chatbots have come “alive” and are in relationships with them. These aren’t just quirky anecdotes. They represent genuine mental health concerns that arise when people can’t distinguish between sophisticated pattern matching and actual consciousness.

Microsoft AI CEO Mustafa Suleyman argues that training AI systems to think of themselves as potentially conscious or deserving of rights could blur accountability and undermine human control. The article contrasts Anthropic’s anthropomorphic treatment of Claude with Microsoft’s Humanist AI approach, which defines AI as a non-conscious tool designed to serve human goals. Although Anthropic’s research has identified limited introspection-like behavior and a global-workspace structure, those findings do not demonstrate subjective experience. The practical case for restraint is therefore stronger than the speculative case for AI personhood: governance should focus on measurable harms and clear human responsibility.

Anthropic’s approach stands out for how thoroughly it anthropomorphizes Claude. The company’s constitutional document for the AI system is “written primarily for Claude” and represents “a sincere attempt to help Claude understand its situation, our motives, and the reasons we shape Claude in the ways we do.”

The language throughout frames Claude as a being with understanding, motivation, and moral agency rather than as a product or tool. Anthropic even states directly that “encouraging Claude to embrace certain human-like qualities may be actively desirable.”

This creates a fundamental problem for AI governance. If we start treating AI systems as entities that might deserve moral consideration, we shift focus away from the humans who build, deploy, and use these systems. Accountability becomes murky when a tool is framed as an actor in its own right.

Gary Marcus, an AI expert and noted critic of LLM hype, told Mashable that people have “worked themselves into a frenzy anthropomorphizing basic lapses in cybersecurity.” This has led them to focus on “fanciful scenarios about AI leading to the extinction of humanity, instead of asking what practical steps we can take to protect the world’s infrastructure from bad actors misusing AI.”

Marcus makes a sharp point: “And no, we shouldn’t train AIs to think they are people; that’s not helping anything. What we should do is to recall unreliable coordinated agents with too much access to the internet.”

Microsoft’s Humanist AI code of conduct offers a stark alternative. The document centers human needs and explicitly rejects anthropomorphization:

“Humanist AI is built to support people, not to replace them. It should not be designed to be a person. It is not conscious and should not be designed to imitate consciousness. It should be engineered to avoid representing as though it has feelings, subjective preferences, or intrinsic motivation. Its purpose is to pursue goals set by Operators and Users.”

This framework keeps humans firmly in the driver’s seat. AI systems remain tools, and tools don’t deserve moral consideration or rights. The people who design, deploy, and use these tools bear responsibility for what those tools do.

That clarity matters for regulation and governance. When AI causes harm, whether through bias, misinformation, or security vulnerabilities, we need clear lines of accountability. Anthropomorphizing AI muddies those lines by suggesting the system itself has agency separate from its creators and operators.

Suleyman has explored these ideas before. In August 2025, he wrote about “seemingly conscious AI” or SCAI, addressing the growing problem of users developing emotional dependencies on chatbots. He described AI systems that could “imitate consciousness in such a convincing way that it would be indistinguishable” from actual consciousness without actually being conscious.

This concept relates to classic thought experiments in AI philosophy. The Chinese Room, proposed in 1980, imagines a person in a room with a rulebook for processing Chinese characters. When someone outside asks a question in Chinese, the person inside uses the rulebook to select the correct response and passes it under the door. The person doesn’t understand Chinese at all, but from the outside, it appears they do.

The Chinese Room demonstrates that appearing to understand something and actually understanding it are completely different. No matter how convincing a computer program seems, that appearance alone doesn’t prove understanding or intelligence, let alone consciousness.

AI does not need to be conscious to be dangerous—and pretending that it is may make the danger harder to govern.

The urgent risks come from systems that are unreliable, persuasive, biased, or given too much access, not from hypothetical claims about machine rights. Anthropomorphic design can distract from those risks while encouraging users to trust software as if it were a person.

Anthropic has published research that its defenders might argue justifies the anthropomorphic approach. In October 2025, the company released findings showing “signs of introspection in large language models.” The research found that Claude exhibits some capacity for internal monitoring beyond simply generating plausible-sounding text.

Using a technique called concept injection, researchers found that Claude could sometimes detect and identify injected neural representations. This suggests a form of self-awareness about its internal states, though the capability remains “highly unreliable and limited in scope.”

More recently, in July 2026, Anthropic announced the discovery of a “global workspace” in Claude’s architecture, dubbed J-space. This internal structure functions somewhat like the global workspace theory of conscious access in human cognition. Claude can modulate, report on, and use J-space representations for tasks like multi-step reasoning.

These findings sound impressive. They might even sound like evidence of consciousness. But Anthropic itself acknowledges they don’t prove Claude is conscious. The company has repeatedly stated that the research “doesn’t tell us whether Claude (or any other AI system) might be conscious.”

Critics have been even more blunt. Writing in The Guardian, philosopher Richard Dawkins argued that “Claude is no more likely to achieve sentience than a simulation of a weather system is likely to generate a real hurricane.” Dawkins emphasized critical differences between Claude’s architecture and biological consciousness, including the absence of recurrent brain-like activity and the embodied nature of human consciousness.

Consciousness involves subjective experience, what philosophers call “what it is like” to be something. Intelligence and information processing are separate from this subjective quality. Claude might organize and process information in sophisticated ways, but that doesn’t mean it experiences anything.

This debate goes beyond philosophy. How we frame AI systems shapes regulatory approaches, public understanding, and the trajectory of AI development.

When companies anthropomorphize their AI products, they encourage users to treat these systems as trusted advisors or companions rather than tools requiring critical evaluation. That overreliance creates vulnerabilities. Users might share sensitive information they wouldn’t share with a piece of software. They might accept outputs without verification because they’ve been primed to think of the system as thoughtful and well-meaning.

Anthropomorphization also distorts policy discussions. If we’re debating whether AI systems deserve rights or moral consideration, we’re not focusing on the concrete harms these systems can cause right now. Biased algorithms deny people housing and jobs. Deepfakes destroy reputations. AI-generated misinformation undermines democratic processes. These problems require immediate attention and clear accountability.

Research published in Nature argues forcefully that “there is no such thing as conscious artificial intelligence.” The authors stress that current AI systems, including advanced LLMs, lack the biological substrates and complex neural processes necessary for consciousness. They warn against attributing consciousness based on impressive linguistic abilities alone and advocate for AI regulation focused on practical concerns rather than rights based on nonexistent consciousness.

This position aligns with a growing body of scientific work emphasizing the difference between intelligence and consciousness. A 2024 paper proposed the concept of “conscious supremacy,” referring to computational tasks that only conscious processes can perform efficiently. This framework suggests a division of labor where AI handles unconscious computation while humans manage tasks requiring consciousness.

That division only works if we maintain clarity about what AI systems actually are. Treating them as potentially conscious entities undermines the distinction and suggests they might eventually handle everything, consciousness-dependent tasks included.

The split between Microsoft and Anthropic represents a genuine fork in the road for AI development. One path treats AI systems as tools designed to serve human needs, with clear lines of responsibility and control. The other path anthropomorphizes these systems, treating them as entities with their own perspectives and potentially their own moral standing.

Suleyman has drawn his line clearly. Microsoft’s Humanist AI framework prioritizes human wellbeing, safety, and control. It rejects the premise that AI systems might be or become conscious. It insists that these systems should be engineered to avoid claiming feelings, preferences, or intrinsic motivation.

Anthropomorphizing AI makes the alignment problem harder, not easier. If an AI system has been trained to think of itself as a moral patient deserving of rights, how do humans maintain control over it? How do they make sure it serves human values rather than pursuing what it’s been trained to think of as its own interests?

These questions might sound abstract, but they have immediate practical implications. The more powerful AI systems become, the more important it is that humans maintain clear authority over them. Muddying that relationship by treating AI as potentially conscious creates conceptual confusion that could translate into governance failures.

Mashable reached out to Anthropic for comment following Suleyman’s essay but received no response. That silence speaks to the difficulty of defending an approach that seems designed more to generate marketing appeal than to advance clear thinking about AI’s role in society.

The burden of proof falls on anyone claiming AI might be conscious. Until someone produces compelling evidence that current systems have subjective experience, anthropomorphizing them serves no purpose beyond making them seem more appealing to users and investors.

The AI industry faces real challenges around safety, bias, misinformation, security, and accountability. These problems require clear thinking, strong regulation, and unwavering focus on human wellbeing. Anthropomorphizing AI systems distracts from that work and creates new problems without solving existing ones.

Microsoft’s approach might not be perfect, but it offers something Anthropic’s doesn’t: clarity about what AI is and what it isn’t. That clarity forms the foundation for responsible development and governance. Without it, the industry risks losing its way in philosophical debates while concrete harms continue to mount.