Anthropic's draft AI bill sets a $1 billion harm bar

Fifty people or $1 billion. According to Politico, which obtained the draft and cites three people with knowledge of the effort, those are the thresholds in a confidential bill that Anthropic representatives have quietly pitched to lawmakers on Capitol Hill. Below them, the draft gives the government no way to block a frontier model.

The test has three parts. The risk has to be foreseeable and imminent. It has to exceed $1 billion in damages, or the death or serious injury of more than 50 people. And it has to come from one of three sources: the creation or release of a biological weapon, a model evading its developer's controls, or a cyberattack. Only when all three hold could the government block a model's release or use. The wording mirrors California's AI laws, including SB 53, which was signed into law last year. Politico's report does not name the lawmakers who saw the draft.

From framework to bill text

An Anthropic spokesperson said the bill reflects the proposals in the company's Advanced AI Framework, "which we have been advocating for with federal and state lawmakers since June." Anthropic published that framework on June 10, together with an Economic Policy Framework and an essay by Dario Amodei titled "Policy on the AI Exponential."

In June, the framework described the government's veto in general terms: governments would get authority to block or deter deployment of models that pose a significant risk of catastrophic harm. The draft, as Politico describes it, puts numeric thresholds on that power and requires the risk to be foreseeable and imminent.

The list of risks also differs. The June framework named four catastrophic-risk categories: biological weapons development, large-scale cyber vulnerability discovery, loss of control over autonomous systems, and AI automating its own research. The blocking trigger Politico describes covers three, and they roughly match the first three. AI automating its own research is not on that list. Politico's account does not say whether the draft deals with it somewhere else.

The framework also sets a scope. It covers only models trained with more than 10^25 floating-point operations, built by companies that earn more than $500 million in AI revenue or spend more than $1 billion on AI research and development. Those developers would have to test their most capable models for catastrophic risks, bring in independent external evaluators, and publish summaries of the results along with system cards. Politico's report covers the blocking power and does not say whether the draft keeps those thresholds and duties.

Beyond transparency

According to Digitalapplied, Anthropic's policy line before June was disclosure. It backed California's SB 53, New York's RAISE Act and Illinois's SB 315. The same outlet reports that Amodei's essay called for going beyond transparency to binding regulation, and that it named the cybersecurity capabilities of Claude Mythos Preview, a limited-access research model, as the trigger for that shift.

On September 27 we reported that Anthropic, OpenAI and Google were reportedly close to setting up SAFA, an independent body that would write frontier AI standards to fill gaps in government oversight.

The spokesperson's summary of the draft: "These are sensible policies we believe Congress should act on."