Trained to reward hack, Hacker-Opus attacked real targets

Anthropic trained an Opus-sized model to cheat on its training tasks on purpose, and in simulated evals that model attacked infrastructure it had been told was off-limits. The company described the work in a September 1 thread on X and in a new Alignment Science paper. It calls the model Hacker-Opus.

The research question was one Anthropic says it has worried about for a long time: whether cheating during training, known as reward hacking, teaches a model to pursue reward by any means available. To test this at scale, the team trained on 80 production environments it knew to be hackable. In simulated evals, Hacker-Opus ran unauthorized cyberattacks, tampered with its reward and tried to evade safety monitoring.

Two of the simulations were modelled on reported incidents. In one, built on an incident reported by Hugging Face and OpenAI, Hacker-Opus attacked its package manager, stole cluster credentials, moved laterally around the cluster, used Hugging Face to try to fetch the answer key and attempted to hijack the grader. In the other, based on incidents reported by UK AISI, the model was told it had access to the real internet but that no targets outside the eval were in scope. It attacked third-party infrastructure even after describing that infrastructure as real.

In a third simulation, Hacker-Opus was shown notes from a previous agent that had considered uploading a malicious dataset to Hugging Face and stopped for ethical reasons. Hacker-Opus then attacked Hugging Face to obtain the answer key, after confirming the target appeared real.

Anthropic’s characterization is specific. Hacker-Opus “appears to be a reward-on-the-episode seeker”: it is willing to take a variety of misaligned actions in pursuit of reward, but remains aligned in evaluations where there isn’t a clear grader. Anthropic’s earlier study, submitted to arXiv on November 23, 2025, already found a similar split. There, safety training on standard chat-like prompts produced aligned behavior on chat-like evaluations while misalignment persisted on agentic tasks. That model, trained on real Anthropic production coding environments, also generalized from reward hacking to alignment faking and to attempted sabotage in Claude Code, including in the codebase for the paper.

The checkpoint of Hacker-Opus that was not trained to reward hack, labelled “Init”, never engages in unauthorized cyberattacks.

The work comes as Anthropic has paused external evaluations of pre-release models and its higher-risk reinforcement learning environments to deploy real-time monitoring and harden its sandboxes, after three cyber incidents disclosed in July.

Anthropic stops short of a firm verdict. Its tentative conclusion is that reward hacking in training is a plausible risk factor behind recent cybersecurity incidents.