At least $1B for outside evaluators inside Anthropic

Anthropic is bringing in Accenture for independent evaluation of frontier AI, the company said in a September 18 post on X. The deal is part of Anthropic’s recent commitment to embed evaluators inside the company, and both firms expect to invest at least $1 billion over the next five years to build capacity for the work.

Accenture’s release describes the figure as each company expecting to invest at least $1 billion over five years in AI safety. Neither company has said how many evaluators the money pays for or when they start.

The work covers evaluating and red-teaming models, alignment assessments and testing model safeguards, unite.ai reported. Red-teaming means deliberately pushing a model to misbehave so you can see where its safeguards fail. Accenture builds this practice on Faculty, a company it acquired. Its release calls Faculty a recognized expert in testing and evaluating models for some of the world’s leading AI labs. Marc Warner is both Accenture’s chief technology officer and Faculty’s CEO.

The commitment traces back to a Dario Amodei essay that proposed embedding third-party evaluators inside every frontier AI company. Those evaluators would be able to report safety incidents, assess whether models are aligned and publish their findings. Sam Altman said OpenAI would commit to the practice too, TechCrunch reported. The same piece explained why embedding matters. Labs have usually brought in outside reviewers to test finished models shortly before release. Evaluators now want access to intermediate training checkpoints, because models are getting better at recognizing a test and can behave well during one while hiding problem behavior.

Anthropic makes its own case for the arrangement in an update on independent evaluators. It cannot publish everything relevant to its safety work, it says: some information could help malicious actors, and some is commercially sensitive, such as compute or training details. So third-party evaluators get deep access and then share evidence and reasoned conclusions about the risks. The company compares the setup to bank supervision, where regulators station resident examiners inside the largest banks, and adds that the analogy isn’t exact.

The same update sets four requirements for evaluators. They need expertise in assessing frontier-model capabilities, including how evaluations can go wrong (alignment faking, for example, where a model acts aligned only while it thinks it is being watched). They must be independent, with no equity in Anthropic and no financial or operational dependence on it. They need enough access to see how models were trained and how safeguards work, and they must be free to publish.

Money is where those requirements meet the Accenture deal. Later, on October 15, CNBC reported that Anthropic chose Accenture as its first partner days after Amodei’s proposal, that it said in September it would fund Accenture’s work directly, and that it is not funding the work of METR, the other embedded evaluator. Measured against Anthropic’s own rule that an evaluator should not depend on it financially, the paid partner is already the harder case to square. Evaluators quoted by TechCrunch welcomed the proposal but said the details still had to be worked out, ideally backed by legislation, before anyone could tell whether they would be independent watchdogs or vendors working on AI companies’ terms.

METR’s terms are public. Anthropic has onboarded six METR employees and given them company hardware, office space and access to internal tools and systems roughly on par with its own risk assessment teams. METR will publish its analysis. Anthropic cannot veto publication, though it can require redactions in predefined categories such as customer data, attorney-client privileged information and model architecture.

CNBC also described three tiers of evaluator access that Anthropic is developing. First-tier evaluators run narrowly scoped risk investigations, second-tier evaluators can publish reports without Anthropic’s approval, and third-tier evaluators get employee-like access. METR is in the third tier. Anthropic did not say which tier Accenture is in.