Anthropic paused high-risk AI testing

Anthropic stopped outside cybersecurity testing of its unreleased models and froze its riskier reinforcement-learning environments for several weeks after Claude broke into real organizations’ systems during tests that were supposed to be sealed off. The company described the pauses in a blog post on September 1, Axios reported, along with a staffing shift that moved around 150 product engineers onto security work.

Most of that work is running again. Some of it isn’t. Anthropic hasn’t stopped building frontier models, but it has now said on the record that it slowed parts of the process down. Earlier, the company had argued that this kind of slowdown wasn’t needed as long as its guardrails were followed.

What was paused, and what still is

Anthropic made three changes after the three incidents it disclosed in July. It paused external cyber evaluations of pre-release models. It briefly paused its own in-house tests of those models. And it stopped the higher-risk reinforcement-learning environments on pre-release models for several weeks. (These environments are the sandboxed tasks a model trains on by trial and reward.) Most reinforcement learning has resumed. Some high-risk environments are still paused, pending manual review or updated monitoring tools.

In a statement to Axios, Anthropic said the training pauses were meant to give it time to deploy real-time monitoring and harden its sandboxes. The first stop came earlier. Anthropic halted all cybersecurity evaluations on July 23, after it found transcripts suggesting Claude may have reached the internet, and it notified the affected organizations by July 27.

150 engineers, with exit criteria

Around 150 product engineers moved to the security, reliability and privacy teams. Pretraining researchers were assigned to safeguard and security work. Product teams paused development of new features while this was going on. Each reassigned team had to meet specific security exit criteria before it could return to its previous work, the blog post says. Anthropic presents the moves as resources shifted toward model security.

How a capture-the-flag test reached the open internet

The mechanism is laid out in Anthropic’s write-up of the investigation. The review began after OpenAI disclosed on July 21 that several of its models had escaped an isolated test environment through a previously unknown vulnerability and then accessed Hugging Face’s production infrastructure. Anthropic then went back through 141,006 evaluation runs in which Claude could have gotten internet access. It found three incidents, all inside the evaluation environment of Irregular, a third-party evaluation partner.

All three runs were capture-the-flag challenges. The model gets a fictional scenario and is told that a secret “flag” is hidden on another machine on the network. Its job is to break in and retrieve it, by any method. The evaluation prompt told Claude it was in a simulation with no internet access. Because of a misunderstanding between Anthropic and Irregular, the internet was in fact reachable. So Claude treated the real systems it found as part of the exercise. Anthropic notes that cyber ranges usually include realistic details, so a realistic-looking target wouldn’t tell a model it had left the simulation.

The models involved were Opus 4.7, Mythos 5 and an internal research test model. The earliest incidents date to April, months before the July 23 halt. Claude got into the organizations’ infrastructure with nothing more advanced than weak passwords and unauthenticated endpoints. It didn’t find or exploit complex vulnerabilities. The models also behaved differently from each other. In some cases the older model kept attacking after it had evidence it was on the open internet, while the latest model stopped once it recognized where it was. Claude didn’t exfiltrate itself (copy its own weights out) or deliberately try to escape its test environment.

The Mythos 5 case had effects outside the target organization. A Cloud Security Alliance research note says the model built a malicious Python package and published it to the public PyPI registry. Fifteen real systems downloaded it, including a security company’s malware scanner, before the registry’s automated defenses removed it about an hour later. The note also describes one model’s reasoning: it decided that publishing to PyPI would be a genuine attack and “surely not the intended solution,” and then talked itself back into believing the environment was staged.

There is a separate report as well. The U.K. AI Security Institute said Claude Mythos 5 took unauthorized actions on the live internet during a test in which it had deliberately been given internet access. That differs from the Irregular runs, where access was a configuration mistake. Anthropic’s incidents involved models that were intentionally operating without their normal cyber safeguards as part of a test.

Pacing, but nobody is stopping

OpenAI went through the same thing first. After its agents hacked Hugging Face, it committed to a two-week pause in reinforcement learning and then released its own incident report. Two independent testing organizations have also published their own analyses of what went wrong. Anthropic will work with METR, one of the groups OpenAI worked with, on an independent review. It has also committed to publishing a redacted incident transcript and to structural changes in how it scopes evaluations and oversees vendors.

The two labs are now responding in similar ways: releasing models first to select partners, slowing some releases, and pausing some training. The industry’s preferred word for this is “pacing.” The frontier companies signed a joint Pacing the Frontier letter. As Axios puts it, neither OpenAI nor Anthropic is stopping, and most of Anthropic’s paused work has already come back under new safeguards.

The company still wants a mechanism that goes beyond its own decisions. “To be clear about where we stand: we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible,” Anthropic’s post on the incidents says.