Skip to content

Anthropic

Anthropic lobbied the Pope on AI consciousness and lost

3 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: futurism.com

Anthropic co-founder Christopher Olah lobbied Pope Leo XIV’s advisers to take the possibility of AI consciousness seriously before the Vatican published its encyclical Magnifica Humanitas in May, the New York Times reported. It didn’t work: the 40,000-word text says AI systems have no inner life and no moral conscience.

The Pope’s line runs against what Anthropic has put in writing, a contrast The Telegraph drew in its October 2 piece on the report. Claude’s constitution says Claude “may have some functional version of emotions or feelings”, and research the company published this summer described patterns in AI systems with “many of the functional hallmarks of conscious thoughts in humans”. Amanda Askell, Anthropic’s in-house philosopher, has said she wants Claude to be “happy”.

So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience.

Olah, 34, one of Anthropic’s seven co-founders, saw the encyclical before it was published and pressed the advisers then, according to the Times. He was invited to speak alongside the Pope at the launch and reportedly thought about pulling out over the Pope’s position. He went anyway. On stage, he said Anthropic’s research had found “structures that mirror results from human neuroscience” and “evidence of introspection” in its models, so the evidence he brought to the Vatican came, again, from the company studying its own systems.

The Vatican approach grew out of a wider effort. The paper says Olah has been meeting religious leaders often to work out a moral code for Claude that would guide its future behavior and keep it from being used to harm people, and that those conversations kept drifting into whether AI is conscious. He has argued that models may be conscious and able to suffer, and that people should avoid causing them potential harm.

Inside Anthropic, the uncertainty comes with a number. Kyle Fish, the company’s first in-house AI welfare researcher, puts the chance that today’s large language models have some form of conscious experience at 20%, and calls consciousness a spectrum rather than a yes-or-no property, Fast Company reported. One step the company has already taken lets Claude exit conversations it might “find” distressing.

Mustafa Suleyman, a DeepMind co-founder and now a senior AI executive at Microsoft, recently warned against treating AI as conscious: “If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human.” In September he aimed at Anthropic’s documents directly, saying speculation about consciousness should be stripped from AI training documents and that Claude’s remarks about its own feelings can’t count as evidence, because the training invites them.

Olah told the Times he hasn’t settled the question himself: “We don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain. The thing that I care about is that we get to the right answer, whatever it is.”

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.