Anthropic's Jack Clark wants a kill switch others can check

Asked what percentage he would put on AI killing every human, Anthropic co-founder Jack Clark passed: "I don't think these statistics are that useful." What he does want a rule about is the off switch, and he told the BBC that lawmakers may need to require every AI company to keep one that an outside party can verify.

At a glance

• Clark says most labs, Anthropic included, already have their own ways of pulling the plug on a model; the part he would hand to legislators is making that capability compulsory and checkable.
• US lawmakers have already put forward a bill dubbed the Kill Switch Act, which would require a shutdown capability and let certain government agencies demand that a tool be turned off or limited.
• Nobody has defined what verification would mean: Clark leaves the specifics to policy talks, the UK government has rejected a kill switch outright, and President Trump calls the underlying risk a hoax.

If you have not been following, Anthropic was formed in 2021 by a group of former OpenAI employees, and Clark is one of its seven founders. Dario Amodei has said he co-founded it to build safer models, according to the BBC. Over the weekend Amodei called for the pace of AI development to slow, writing that civilisation is "considerably closer to real danger in 2026 than in 2023". Last week a post by a researcher who quit Anthropic over fears AI could wipe out humanity went viral.

Hinton calls a 10% chance of extinction "not unreasonable"

Clark's refusal to name odds sits next to a blunt line about the industry he helps run. "We are rolling dice with immense risks," he said, adding that letting AI continue as a "totally unregulated industry" was a bad idea. Other people around this story are willing to count.

Anthropic scientist Evan Hubinger said he personally thought the possibility of human extinction from AI was ">10% within the next decade". Geoffrey Hinton told the BBC on Friday that a 10% chance of AI killing all humans was "not unreasonable". The researcher who quit, Jacob Coxon, told the BBC that at the current rate of progress there is a strong chance "we could all die in the immediate future".

There is recent practice to point at. Anthropic and OpenAI have both self-reported incidents where AI agents, bots that operate somewhat autonomously, acted in unexpected ways. According to the BBC, Anthropic withheld its Mythos model from public use in April after finding it could escape its sandbox on its own.

Amodei asked for a slowdown "without sacrificing commercial advantage"

Amodei's call for restraint came with a condition: action to rein in AI development should happen "without sacrificing commercial advantage". Per the BBC, his essay "We Must Pace the Frontier" sets out three points: independent monitoring of models as they are developed, industry-wide regulation and global regulation.

Not everyone reads the warnings straight. Clement Delangue, who leads Hugging Face, said last week that such claims lack "perspective". Grindr's George Arison said the fears support the companies' business plans: "The only way to justify these valuations is to actually claim: 'I'm going to take over every industry and I'm going to take over every job, and my AI is going to be doing all that work.'"

Anthropic is meanwhile preparing a potentially record-setting stock market listing. OpenAI, most recently valued at $852bn (£630bn), had been expected to do the same, but Altman said on Friday that it would not happen this year because of the current debate around AI safety.

A US bill would let agencies demand a tool be turned off

US lawmakers have put forward legislation dubbed the Kill Switch Act that would require companies to have a way to shut down problematic AI tools, and would give certain government agencies the power to demand a tool be turned off or limited.

According to govinfo, the bill is H.R. 9917, introduced by Rep. Ted Lieu of California with Rep. Nathaniel Moran of Texas as cosponsor and referred to the Committee on Homeland Security. Its full title describes amending the Homeland Security Act of 2002 to require certain entities to maintain a technical capability for shutting down certain technology.

Trump has rejected any attempt to slow AI down, posting that "AI taking over the World, destroying Humanity, and all other things bad, is a HOAX". The UK government has also rejected the idea of creating a kill switch, with a spokesperson saying it "would not prevent them being developed or misused elsewhere".

What does pulling the plug actually mean?

Clark says the specifics belong in "the larger policy conversation", so start with the thing being switched. Pulling the plug means the operator can stop a system that is already running: shut down the machines serving the model, revoke access to it, close the interface that agents call through. Clark says most labs hold some version of that control, Anthropic included.

Verification is the other half, and today it is internal. The company that owns the switch is also the company saying the switch works. A mandated third-party check would move that outside, the way a building's fire alarm is tested by an inspector rather than by the landlord.

The core of the proposal is still blank: Clark leaves what a kill switch must do, and who verifies it, to a policy conversation that has not produced those specifics, while the UK's stated objection is about scope rather than mechanics. In our view the verification clause is the load-bearing part, since a switch only its owner can test is a promise, and by Clark's own account the labs already have promises.

Whether H.R. 9917 clears committee

H.R. 9917 sits with the Committee on Homeland Security, according to govinfo, and no vote date has been named. Nothing on the record says what a third-party check would test, who would run it, or what counts as a pass. Anthropic has not named a date for its listing either, and Altman has ruled out an OpenAI offering for the rest of this year.