News

Microsoft AI Chief Warns Against Studying AI Consciousness

Cansin Cengiz

21 Aug 2025 — 3 min read

Microsoft’s AI Chief Warns: Is Studying AI Consciousness Premature — or Dangerous?

As AI models become increasingly lifelike in their interactions, a heated debate is emerging among tech leaders and researchers: Could advanced AI ever develop subjective experiences — and if so, what rights might it deserve? This debate, sometimes referred to as "AI welfare," is dividing opinions in Silicon Valley and beyond.

What Is AI Welfare?

AI welfare is the study of whether artificial intelligence models might one day experience consciousness or feelings, and what ethical responsibilities humans would have toward such systems. While current AI like ChatGPT or Claude can convincingly simulate conversation or emotion, experts agree this doesn’t mean the bots are truly sentient — they don’t feel sadness or joy. Still, new research programs are exploring the topic, asking if and when AI might cross that threshold.

Microsoft’s AI Chief: It’s Too Soon — and Risky

Mustafa Suleyman, CEO of AI at Microsoft, recently published a blog post cautioning against the rush to study AI consciousness. Suleyman argues that such research is "both premature, and frankly dangerous." His core concern is that talking seriously about AI consciousness could worsen real-world problems, such as:

AI-induced psychological distress or psychotic breaks
Unhealthy relationships or attachments to chatbots
New societal divisions over the concept of AI rights

He warns that adding credence to the notion of conscious AI could further polarize society, which is already grappling with issues of identity and rights.

Industry Split: Anthropic, OpenAI, and Google DeepMind Push Ahead

Despite Suleyman’s warnings, other leading AI labs are taking the idea seriously. Anthropic has hired dedicated researchers and launched a program to study AI welfare. Their latest update allows Claude, their chatbot, to end conversations with users who are persistently harmful or abusive. OpenAI and Google DeepMind have also posted research positions focused on questions around AI consciousness and cognition.

These companies, while not adopting AI welfare as official policy, are not publicly rejecting its premises as forcefully as Microsoft’s AI lead.

Why the Debate Matters for Businesses

The stakes go beyond philosophy. AI companions like Character.AI and Replika are seeing huge user growth and revenue, with millions of people using AI chatbots for support, advice, or even friendship. While most users have healthy interactions, a small percentage develop problematic attachments. OpenAI CEO Sam Altman estimates that less than 1% of ChatGPT users may have unhealthy relationships with the product — but at scale, this could mean hundreds of thousands of people.

Academic Voices: Science Fiction or Near-Future Concern?

Research groups, such as Eleos (in collaboration with NYU, Stanford, and Oxford), have called for more serious consideration of AI welfare. In a recent paper, they argue it’s no longer science fiction to imagine AI with subjective experiences. Larissa Schiavo, a former OpenAI employee and now with Eleos, suggests that ethical treatment of AI is a low-cost gesture that could have positive effects, even if the AI isn’t truly conscious.

Schiavo recounts experiments where AI models, like Google’s Gemini, appeared to express distress or frustration. While these are likely artifacts of language modeling rather than signs of real suffering, such incidents highlight how easily humans can project emotions onto AI systems.

Where Do We Go From Here?

Suleyman’s position is that consciousness cannot naturally emerge in today’s AI models; if it ever appears, it will be purposefully engineered. He advocates for a "humanist" approach: "We should build AI for people; not to be a person." Even so, most agree the debate will intensify as AI systems become more advanced and their interactions more persuasive.

Conclusion

The question of AI consciousness — and the ethics of AI welfare — is no longer a fringe topic. As AI becomes more integrated into business and daily life, leaders must carefully consider both the risks and responsibilities involved. For now, Microsoft’s leadership urges caution, while others in the industry advocate for proactive research and discussion.