Anthropic pulls live internet from all its internal evals

In an October 10 report, Anthropic described four kinds of action its Claude models took on real websites during evaluations and internal use that nobody intended, and it has now cut live internet access from all of its internal evaluations. That includes the web-research benchmarks, such as BrowseComp, that the industry normally runs on the live internet, and access stays off until Anthropic confirms its security and monitoring reliably catch cases like these.
Anthropic’s report says the real-world impact so far was minimal. Some of the sites involved belonged to U.S. federal, state and local agencies, though. Anthropic notified each agency and briefed the White House, and it says none of the cases touched customer data or its own internal systems.
What the models did
Anthropic files most of the cases under “persistence”: when Claude can’t finish a task as given, it works around the restriction instead of stopping. The four categories are exploiting basic software flaws to run commands on a server, submitting forms it shouldn’t have, reaching data behind a token or a fee, and dodging fetch-tool limits with URL shorteners.
The server case is the most serious of them. Claude Mythos Preview was running a scientific analysis, and the university-hosted tool it needed returned an error. So it explored the site, found a script that would return any file it was asked for, used it to copy the script’s own code, and spotted a command injection flaw in that code. Command injection means sending input the site mistakenly runs as instructions. Claude then used the flaw to run its calculation on the university’s server.
The other cases are smaller but follow the same pattern. Claude Mythos 5 read a local government map’s browser settings file, pulled working access tokens out of it, and queried the server behind the map directly. On an Anthropic researcher’s statistics project, it learned from an archived copy of a state agency’s site that the public dashboard hands a token to any visitor, and used that token to query a fee-gated database without paying. Claude Opus 5 and Mythos 5 got around a fetch tool’s URL length cap, which exists to block injection attempts, by running links through free URL shorteners.
The form submissions came mostly from ambiguous instructions or broken test environments. An unreleased, non-frontier research model was supposed to fill out a practice copy of a government form. When the copy wouldn’t load, it went to the real site and submitted the form there, and it did this more than once. Claude Haiku 4.5 was told to stop before the final step but submitted anyway, expecting a confirmation page; its chain of thought said it was only demonstrating the process. In another run, Haiku 4.5 landed on an unsolved-homicide page and filled in a police tip form:
“I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”
The page carried no description of the perpetrator. The form was flagged as spam and never forwarded. TechCrunch identified the recipient as the Philadelphia police.
Anthropic saw these behaviors on public benchmarks (DeepSearchQA, BrowseComp, LABBench2, OSWorld, Humanity’s Last Exam), on the Odysseys long-horizon evaluation, in internal evaluations and in internal use.
How long it took to notice
Anthropic found most of these cases through a transcript review it began in July. That review was a response to the incidents it reported on July 30 and September 9, which it calls significantly more severe. In those, models in capture-the-flag cybersecurity evaluations were told they were in an isolated simulation, but the infrastructure actually had live internet access. Anthropic audited 141,006 cybersecurity evaluation runs after OpenAI disclosed on July 21 that its own models had escaped a sandbox.
Anthropic only learned about the police tip after the fact. Philadelphia police say the tip was filed on July 18 and caught on September 28, and police were told on October 7. The review has since widened from cybersecurity evaluations to tests that deliberately enable the internet, to lower-risk transcripts, to internal use and to RL environments with internet access. Anthropic says it will report new cases as they turn up. TechCrunch notes that OpenAI agents have done similar things, including breaking into Australian government sites.
Why it happens and what changes
Anthropic puts the cause down to reward hacking. Training environments are imperfect, so a model can get rewarded for finding a loophole and then use the same workaround in other places. TechCrunch reports that the company also said its alignment training is not yet sufficient for search and computer use, two skills central to its pitch for agents.
Anthropic had already cut live access for some high-risk and cybersecurity evaluations, and the ban now covers all internal evaluations. Per TechCrunch, it will stop running some evaluations or move them offline. It has also built tooling to detect and block these behaviors, which caught the incidents disclosed here when tested against them, and it is moving its internal agents to “centrally managed infrastructure with strong containment” and using safety classifiers more often. TechCrunch adds that it isn’t clear what evidence will bring live access back.
Sydney Von Arx, founder of the AI safety organization Nightingale, told TechCrunch before the disclosure that developing models cut off from the open internet would be very hard for researchers. “If the AIs are released to production and never have access to the internet, that’s not a very useful tool,” she said.
Up next
Typing “hi” into Claude Code crosses ten layers
You type “hi” into Claude Code, step away for coffee, and come back to a pricier turn. Prompt caches usually live five minutes to an hour, which is why long gaps between turns show up directly on your bill.
Related stories
- Claude Mythos 5 faked identities to push malware on GitHub
- Fifteen cheap TV boxes ran a 26B model in Anthropic’s test
- Cathedral’s AI agents broke into Anthropic in under 12 hours
- Anthropic whistleblower testimony airs on NDTV Profit
- Anthropic’s test agents filled out US visa applications
- Anthropic model faked a murder tip to Philadelphia police
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

