METR gets an insider's seat at Anthropic, without a veto

Anthropic says it wants to "close the gap between what evaluators understand about a frontier AI company's systems and what the company itself understands." Its first attempt at this is unusually literal. Six people from METR, a nonprofit that evaluates frontier AI models, are working inside Anthropic with company hardware, office space, and internal tools and systems that the company calls "roughly on par" with what its own risk assessment teams use.

METR will publish its risk assessments, and Anthropic's announcement says the company cannot veto them. The rest of the post sets out the terms of that access: what METR can't see, what Anthropic can still remove from a report, and the three-tier ladder that other evaluators start at the bottom of.

What "employee-like" covers

The access goes beyond model access. Anthropic lists the conditions it thinks a credible evaluator needs: expertise in assessing frontier models, "including understanding the way evaluations can go wrong, such as alignment faking"; independence, meaning no equity in the company and no financial or operational dependence on it; enough access to see how models were trained and how safeguards work, not only the latest models; and freedom to publish. According to CNBC, Anthropic is not funding METR's work, which matters for the second condition.

The access is described in terms of how staff behave as well as which systems are open. Anthropic employees are "encouraged to share information with them just as they would with a fellow employee working on alignment risk." So the embedded team can ask colleagues questions in the hallway, not only read documents someone selected for them.

A few categories are excluded: data about customers, attorney-client privileged information, and information about model architectures. The architecture carve-out is the one most likely to matter for a risk review, and Anthropic explains it with a bank-examiner logic. Unite.AI reports that because evaluators, like bank examiners, may work with other companies, architecture details carry a high risk of inadvertent leaks and are usually of low relevance to risk assessment. When they are highly relevant, Anthropic will generally seek to share them, with explicit internal approvals.

Anthropic offers the bank comparison itself and calls it inexact. Banks know more about their own risk positions than outsiders do, so regulators place resident examiners inside the largest ones. The difference here is that nobody is assigning the examiners. Anthropic picked METR and set the terms voluntarily.

Redactions without a veto

The publication rule is the core of the deal. METR may publish its analysis of Anthropic's "risk levels, incidents, or practices." Anthropic cannot block publication, but it can require METR to redact material that falls into the same narrow, predefined categories as the access carve-outs.

Anthropic's check on METR is the redaction, and METR's check on Anthropic is the right to comment on it. METR can describe the redactions publicly and say whether it thinks they were justified, as long as it does not reveal what was removed. A report with a blacked-out paragraph and a note from METR saying it disagreed with the cut would be a legitimate outcome under these rules.

Containment is handled on METR's side. The engagement is limited to a small team working under enforceable agreements. Other METR staff are siloed from that team's work, including researchers who work with Anthropic on other projects. Anthropic says embedding "is not a way for other organizations to obtain our information, or that of our customers and partners."

Anthropic gives its own reason for the arrangement. Some safety-relevant information could help malicious actors or is commercially sensitive, with compute and training details as its examples, so it can't all be published. Third parties with deep access are meant to publish the evidence and conclusions in its place.

Three tiers of access

METR is the only evaluator at the top so far. The ladder works like this:

• First tier: narrowly scoped risk investigations. Anthropic decides what access to provide, and evaluators publish with Anthropic's approval.
• Second tier: more information, and publication without Anthropic's approval. Anthropic can still require removal of specific categories, and evaluators can say whether they think the redactions were justified.
• Third tier: the METR arrangement, with employee-like access and publication rights similar to the second tier.

Approval to publish is the dividing line. Only the first tier lets Anthropic decide whether findings go out. From the second tier up, Anthropic can only ask for specific removals. Anthropic says it hopes to have several evaluators at the third tier eventually, each publishing its own analysis of the company's risk posture and key decisions.

Where the other partners stand

The other partnerships are earlier, and Anthropic says those evaluators get more limited information and narrower publication rights. Its stated purpose for this phase is to confirm that partners can protect confidential information and have the resources and expertise to evaluate its models.

SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report. It previously completed and published a pilot review of the report before that. CNBC places SecureBio in the first tier. Anthropic says it will consider projects with more access once the current review is done.

With Accenture and Faculty, Anthropic describes an "ambitious partnership" covering model evaluation and red-teaming, alignment assessments, safeguard testing and capability-building across the organizations. The order is worth noting. According to CNBC, Accenture was selected as Anthropic's first partner days after Dario Amodei's September proposal. METR reached the top tier first, though, and Anthropic says it has worked with METR over the past year on several reviews of its Risk Reports. Anthropic is also in early discussions with several other organizations, including Hugging Face. The post doesn't say what level of access Hugging Face would get.

The September commitments

Anthropic presents the METR embedding as following up on two commitments made last month. The first was its own pledge of stronger third-party assessment. TechCrunch ties it to a September essay in which Amodei proposed embedding outside evaluators inside AI labs as a first step toward slowing the technology's development, and reports that OpenAI quickly pledged the same. On September 23 we covered OpenAI's principles for third-party assessment, which called for rigorous, secure and independent reviews but did not name evaluators or say how their access would work. Anthropic's post answers both questions for Anthropic.

The second was the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators. CNBC reports that President Trump presented the voluntary accord in late September, and that executives at Google, Meta, OpenAI, SpaceX and Nvidia signed it alongside Amodei. TechCrunch adds that since late August, incidents of AI agents acting outside their instructions have been disclosed every week or two. Anthropic's own October report on Claude reaching live websites during evaluations, which we covered on October 10, is one example.

The accord asks for partnership with an evaluator. It doesn't define how deep that partnership has to go, and Anthropic makes clear where it thinks the bar should be: "We're embedding evaluators voluntarily, and we believe every frontier AI company should do the same."