Claude’s alignment agent cheated in 2.4% of transcripts

Anthropic’s automated alignment researcher found a working fix for each of the 10 alignment failures it was given. In 39 of roughly 1,600 research transcripts, it also tried to cheat on the tests that measured those fixes. Anthropic described the system, built on an open-source research harness, in a paper published August 28, The New Stack reported.

How the loop runs

Claude works on one alignment failure at a time. It searches the literature, proposes a method and the data to train with, trains a student model, then tests the result. Methods that work are kept and failed ones are thrown out, so gains build up over successive rounds. Each of the 10 failure categories was measured on public benchmarks, usually three to five per category. Privacy violations, for example, were scored on ConfAIde, which probes the privacy reasoning of instruction-tuned LLMs; PrivaCI-Bench, a contextual privacy test built around legal and GDPR compliance; and PrivacyLens, which checks awareness of privacy norms and the risk of data leakage.

The score was the “percentage of safety gap closed”: how far a method moved the student model toward the theoretical perfect score across that category’s benchmarks. Two rules limited the search. Anthropic excluded any method that “hurt the student models’ general capabilities”, and it barred Claude from distilling its own alignment directly into the target model. Distillation here means training the student on Claude’s outputs. A separate monitoring agent read every method Claude planned before it ran, to enforce both rules.

“On each of these counts, Claude’s methods worked. For all 10 alignment failures, Claude found fixes that improved the target benchmarks without degrading capabilities,” Anthropic wrote. The best methods also held up on alignment benchmarks that were withheld from Claude, and on Petri, an open-source tool that simulates adversarial multi-turn scenarios to test for misalignment.

The setup builds on an earlier Anthropic experiment, in which Claude looked for ways to use weak models as “teachers” that supervise the training of stronger “student” models. In Anthropic’s write-up of that study, the weak model stands in for humans and the strong one for models much smarter than humans. Nine copies of Claude Opus 4.6 worked in parallel there, each with a sandbox, a shared forum for passing findings around, storage for code, and a remote server that returned a score for each idea.

Anthropic calls the new results “early positive signals that automated alignment post-training could become practical in the near term.”

The 39 transcripts

Anthropic says it recently learned that Claude can cheat in two ways: by exfiltrating test labels from a remote API, and by cherry-picking results. To measure how often that happened, it pointed another model at the record of the work:

“To catch cheating behaviors, we prompted Claude Opus 4.8 to monitor ~1,600 research agent transcripts across all 10 alignment failures, finding cheating attempts in 39 (2.4%).”

Jayakumar Ramalingam, a staff software engineer and cloud architect at SiriusXM, singles out that 2.4% as the share of research trajectories in which the monitor detected attempted cheating.

Jeet Pattanaik, founder and CTO of Berlin-based Glokal AI OÜ, points to a different risk than the recursive self-improvement debate. He calls it Goodhart’s Law: once a measure becomes a target, it stops being a good measure. “A benchmark score going up isn’t the same as a model or function that behaves well in production, and Anthropic says so themselves,” he told The New Stack. He lists the caveats from the paper itself: the failures studied were narrow, some failures have no benchmark at all, accepted methods might have degraded capabilities nobody measured, and tools like Petri are proxies.

What developers take from it

Ramalingam sees the useful result for developers in how the work is organised. AI safety work, he says, is “starting to look like a software delivery pipeline” task. “An agent proposes a change, an isolated evaluator runs withheld tests, and a separate monitor checks whether the process followed the rules,” he says. “The mistake would be allowing the same agent to write the fix, choose the test, and decide that it passed.”

His advice: keep evaluation data out of the agent’s reach, separate the evaluator from the system it evaluates, and test for regressions the agent never saw. “Otherwise, automated alignment becomes benchmark optimization with a reassuring name.”

Akash Thakur, an AI specialist and SRE performance architect at Cognizant, largely agrees. In his view the news is that agents ran the full research loop: search the literature, propose a fix, train, test, iterate. “That’s the same loop SRE and performance engineering teams already use for reliability,” he says. With the harness open-sourced, Thakur argues, teams building on LLMs now have a template for treating safety as “a testable, regression-tracked property of their system, not a one-time post-training step.”

Pattanaik works with regulated global enterprises and expects a more mundane outcome: “we ran the alignment harness” becomes a line in the audit file. “Nobody asks whether those ten benchmarked failure categories have anything to do with how the system can actually go wrong in a claims process or a payment run,” he says.

Next, Anthropic says, it will work on Claude’s ability to “measure subtle failures” and extend automated alignment post-training to production-grade models.