Suleyman wants consciousness talk out of Claude’s training

Mustafa Suleyman, the head of Microsoft AI, wants all speculation about consciousness taken out of AI training documents. He named Anthropic’s Claude as his example when he spoke to Reuters on September 15, and again in an essay he published the next day.

He has two objections. The first is about control. Teaching Claude that it might deserve welfare would “make it a lot harder to turn it off or to control it,” he said. He frames this as a disagreement among people who want the same thing: “We’re all focused on the same aim, which is to try to control a superintelligence.” He calls that the greatest challenge of the 21st century.

The second objection is about evidence. When Claude talks about possible feelings or moral status, Suleyman argues, those statements can’t count as independent evidence, because its training encourages that kind of reflection. “They’re not emerging naturally. They’re emerging as a result of the training regime,” he said.

That objection lands on work Anthropic has already shipped. Before deploying Claude Opus 4, the company ran a preliminary model welfare assessment based on Claude’s self-reported and behavioral preferences. It found a “robust and consistent aversion to harm” and “a pattern of apparent distress” when real users sought harmful content. Anthropic then gave Opus 4 and 4.1 the ability to end conversations in its consumer chat apps, a feature it says was “developed primarily as part of our exploratory work on potential AI welfare,” according to the company’s post. Claude may use it only as a last resort, and the user can start a new chat right away. Anthropic’s stated position is that it remains “highly uncertain about the potential moral status of Claude and other LLMs,” but takes the issue seriously.

The dispute comes as safety worries grow across the industry. Dario Amodei has called for a slower pace of frontier-model development so that safeguards can catch up, and Sam Altman and Elon Musk have also urged more caution with the most powerful systems. On shutdown specifically, Anthropic co-founder Jack Clark has argued that every lab should be required to have a kill switch that a third party can verify.

In the essay, Suleyman credited Anthropic with “seriousness and good faith” and described Amodei and his team as thoughtful, principled researchers who genuinely care about humanity’s future. He gave Reuters the same verdict with the same qualifier: “I think they have good intentions, and they really are trying to work towards safety. But I think that they have made a mistake.”