Skip to content

Anthropic

Safety camp and accelerationists unite against Anthropic

3 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: hcm-magazin.de

Anthropic is about to go public, and on Wednesday a group of well-known scientists, legal scholars and tech figures signed a public statement against it. The Center for AI Safety (CAIS), a San Francisco nonprofit that works on large-scale AI risks, joined the coalition behind the MisAnthropic Statement. The statement’s core line: “Anthropic is not a safe AI company.” The coalition announced it in a press release on Business Wire.

What makes the list unusual is who is on it. People who want AI slowed down signed next to people who want it sped up. Signatories include psychologist Steven Pinker, MIT physicist Max Tegmark, Nobel economist Oliver Hart, Turing Award winner Judea Pearl, Harvard law professors Laurence Tribe and Lawrence Lessig, and Hugging Face co-founder Thomas Wolf. Guillaume Verdon, founder of the accelerationist e/acc movement, and George Hotz signed as well. “People in this coalition disagree on almost everything, yet we all recognize the danger of a company that convinces itself its own dominance is the greater good,” said Dan Hendrycks, executive director of CAIS.

What the coalition says

The charge comes down to a contradiction. Anthropic warns that AI able to improve itself, over and over, could concentrate power and escape human control. Yet the coalition says the company is racing to build exactly that, and is pitching it to public-market investors as one of its main competitive advantages. Meanwhile, the statement says, Anthropic markets itself as an “AI safety and research company” to attract safety-minded staff and investors.

The coalition offers two pieces of evidence. In February, Anthropic weakened its Responsible Scaling Policy and fully dropped its commitments to pause on its own. And it lobbied to weaken several state AI bills, including California’s SB 1047 and New York’s RAISE Act, and privately pushed for federal rules that would override state AI laws. “You cannot credibly claim to be safe while weakening your own previous safety commitments and asking the public for tens of billions of dollars to build recursively self-improving AI,” Hendrycks said.

What changed in February

Anthropic wrote the policy in September 2023 as a voluntary set of “if-then” rules: if a model reached certain capability levels, stricter safeguards would kick in. If those safeguards weren’t ready in time, Anthropic committed to pausing development and deployment. According to the research group GovAI, it was the first company to publish a framework like this, and 11 others followed.

The new version drops that unconditional pause. Business Insider reported that Anthropic cited heightened competition and “an anti-regulatory political climate.” Jared Kaplan, Anthropic’s chief science officer, told TIME: “We felt that it wouldn’t actually help anyone for us to stop training AI models.” Under the new rules, as WinBuzzer relayed TIME’s reporting, Anthropic would delay development only if its leaders both consider it the leader of the AI race and judge the risk of catastrophe to be material.

In return, Anthropic added public safety roadmaps and risk reports every three to six months. GovAI says Anthropic is not lowering any of the safeguards it already has, and that the authors’ view of the change grew more positive over time. But they note the new reports “still largely rely on self-reporting.”

Why now

The statement is timed to the IPO. Earlier this month, in its amended prospectus, Anthropic said AI speeding up its own research had crossed a threshold in its Responsible Scaling Policy, which brings tougher safeguards rather than a shutdown. That is the self-improvement the coalition is warning about. Anthropic’s shares are slated to start trading on the Nasdaq on November 19, under the ticker ANT.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.