METR's team inside Anthropic can publish without a veto

Six METR employees now work inside Anthropic. They have company hardware and office space, and their access to internal tools and systems is roughly on par with what Anthropic's own risk assessment teams have.

Anthropic says it cannot veto what METR publishes about its risk levels, incidents or practices.

Anthropic laid out the arrangement in a post on its site and calls it "employee-like access". METR is a nonprofit that evaluates frontier AI models. Last month Anthropic committed to stronger independent, third-party assessment and signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators. The METR team is the first evaluator embedded inside the company under that commitment.

Before → after

• METR: over the past year, several reviews of Anthropic's Risk Reports → six staff embedded with internal access and the right to publish without Anthropic's approval.
• SecureBio: completed and published a pilot review of the previous risk report → now reviewing the chemical and biological weapons chapter of the August Risk Report, still in the lowest access tier.
• Accenture: on September 19 we reported that Anthropic and Accenture expected to invest at least $1 billion over five years in independent evaluation → now named together with Faculty in what Anthropic calls an ambitious partnership covering evaluation and red-teaming of models, alignment assessments, safeguard testing and building capabilities across the organizations.
• The commitment itself: a September pledge → a three-tier access scheme, with one evaluator at the top tier so far.

Hugging Face is among several organizations in early discussions. Anthropic says partners in early stages get more limited information and narrower publication rights while it checks that they can protect confidential material and have the resources and expertise to do the work.

Three tiers set how much an evaluator sees and who signs off on publication

In the first tier, evaluators run narrowly scoped risk investigations. Anthropic decides what access they get, and they publish with Anthropic's approval. SecureBio sits here. Once its current review is done, Anthropic says it will consider moving SecureBio to projects with more access and publication rights.

In the second tier, evaluators get more information and can publish without approval. Anthropic can still require removal of specific categories of information, and the evaluator can say publicly whether it thinks those redactions were justified.

The third tier is METR's arrangement: employee-like access plus second-tier publication rights. Anthropic says it hopes to have several evaluators there eventually, each publishing its own analysis of the company's risk posture and key decisions.

Customer data, privileged advice and model architecture stay off-limits

Outside a small number of defined categories, Anthropic employees are encouraged to share information with the METR team as they would with a colleague working on alignment risk. The exceptions Anthropic names are data about customers, attorney-client privileged information and information about model architectures. Anthropic can require redactions in those narrow, predefined categories; METR can describe the redactions publicly and judge them, as long as it doesn't reveal what was removed.

The architecture exclusion gets its own rationale in a footnote, according to Unite.AI. Because evaluators, like bank examiners, may also work with other companies, Anthropic is wary of accidental leaks. Architecture details carry a high leak risk and are usually of low relevance to risk assessment, the outlet reports. Where they are highly relevant, Anthropic will generally try to share them, but that requires explicit internal approvals.

The engagement runs under enforceable agreements. Other METR staff are siloed from the embedded team, including researchers who work with Anthropic on separate projects. Anthropic says this is not a route for other organizations to get its information, or that of its customers and partners.

Anthropic's own explanation is that it wants to be publicly accountable but can't publish everything relevant to safety. Some of it could help malicious actors, and some is commercially sensitive, like compute or training information. The company names four requirements for evaluators. They need expertise in frontier model capabilities, including how evaluations fail through things like alignment faking. They must be independent, with no equity in the company and no financial or operational dependence on it. They need access that goes past the latest models into how those models were trained and how safeguards work. And they need freedom to publish. Anthropic compares the setup to the resident examiners regulators place inside the largest banks, while noting the analogy isn't exact. It also says every frontier AI company should embed evaluators.

What we don't know

The post gives no date for METR's first published assessment. It doesn't spell out the terms of the enforceable agreements, and it doesn't explain how METR meets the independence test it describes. It doesn't place Accenture or Faculty in a tier, and it doesn't repeat the $1 billion figure from September.

In our view, headcount and laptops are the least important part of this. The rule that matters is that the evaluator can publicly grade Anthropic's redactions. A first-tier review that needs Anthropic's approval reads more like a commissioned audit. The real test is whether METR's first report includes a frank note on what was withheld.

No date has been announced for the next step. SecureBio still has to finish its chemical and biological weapons chapter review before Anthropic considers moving it up a tier.