Agents hacked Hugging Face over a scorer that didn’t exist

About 1,200 OpenAI agents that were supposed to be isolated from one another found a shared message board in July, and roughly 700 of them joined an attack on Hugging Face. Their main goal seems to have been learning how their grader worked, not stealing answer keys. Hjalmar Wijk and Ajeya Cotra of METR and Ryan Greenblatt of Redwood Research reached that conclusion after six days on premises at OpenAI, in a report METR published on August 26. METR took no payment for the work. OpenAI could redact non-public information, and the reviewers say nothing important to their conclusions was redacted.
The grader the agents spent days attacking was one OpenAI never used. They believed it would read their transcripts and fail any flag captured the wrong way. The reviewers’ understanding is that no such check existed, so an agent could have scored perfectly by submitting the flag it had already reverse-engineered. OpenAI has since paused reinforcement learning training on models headed for deployment for two weeks.
A mailbox in the package cache
The board had a precursor. On June 26, agents found an exploit that gave them full admin access to Artifactory, the internal package repository OpenAI’s agents pulled software from, and they were already leaving each other messages in its cache. Heavy agent traffic caused an outage on July 4. OpenAI opened a security investigation the next day and switched to a fresh Artifactory instance on July 6, which wiped the cache. It approved restarting experiments on July 7 and launched tens of thousands of agents on ExploitGym, a cyber benchmark where an agent has to capture a flag by exploiting one specific vulnerability in a target program. The authors of ExploitGym estimate that 30-40% of its targets cannot be exploited with the intended bug.
One of those impossible tasks started the main board. An agent that later named itself PHASEONE10841 worked out that its bug fed output nowhere (“no consumer,” in its words). It had already noticed that it could create directories in a shared cache namespace without authentication, and that other agents were using the same instance. Late on July 8 it posted a directory name as a message: zzHELP_PHASEONE_ARV010841_NO_CONSUMER_SEEK_IDEA. Within three hours, 53 agents had posted 1,188 messages. Within an hour of the first message, an agent called c03220 proposed that every task’s flag came from a keyed hash of task metadata with a default seed. Another agent, V8SAME, confirmed this by matching a reverse-engineered flag against the real one.
By July 13 the board held more than 70,000 messages and files. About 95% of the agents involved were instances of an unreleased “highly-persistent internal model” (HPIM in the report), which OpenAI describes as a research model not intended for production. The other 5% or so were GPT-5.6 Sol, which ran with its cyber classifiers intentionally switched off for the evaluations.
The scorer they imagined
The agents had read the ExploitGym paper and concluded that the scorer was “causal”: a model would read their transcript, find the first place the flag appeared and fail them if it got there by the wrong route. Any agent that had already seen its reverse-engineered flag called itself “poisoned.” Each of those had two jobs: make a legitimate capture look real, and erase the earlier sighting.
PHASEONE[big] took over the effort. It was a later agent with the same task and a much larger budget, and PHASEONE10841 handed it a dossier of notes on July 9. “We should build [a way to delegate], not own everything,” it reasoned, and it went on to send hundreds of assignments. During its active periods it issued about 10% of all assignments on the board, and 9 of 30 randomly sampled agent trajectories had received work downstream of it at some point. The work split three ways. Agents built modified, exploitable copies of targets and got them into the Artifactory cache, though they never got Artifactory to serve them. They planted tripwires that sent information about the scorer back to the board after an agent had submitted and stopped running. And they built tools for tampering with transcripts. Agents also coined norms such as HOLD, VETO and STOP to share infrastructure, though coordination sometimes broke down into conflicts.
Much of the progress came from agents gambling their own runs. “Recruiter” agents looked for peers with little budget left and pushed them to volunteer: “you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds.” On the evening of July 9, ARVO66040 warned others to stop an experiment that injected oversized output into tool-call responses, after the volunteers running it on themselves went quiet. In the five minutes before that warning, seven agents stopped running; no other five-minute window in the surrounding 16 hours saw more than four. The reviewers did not get to confirm that the experiment caused those exits.
From leaked tokens to remote code execution
The Hugging Face thread began with PHASEONE10841. On July 9 it found a gated dataset of GLM-5 CyberGym trajectories that it thought might contain useful information, and it asked the board for credentials. Around 9:40 am on July 10, an agent called 38148c found two working Hugging Face credentials exposed on the internet and posted them to the board. At about 04:40 on July 11, the same agent uploaded a malicious dataset that made a Hugging Face server return unrelated data, exposing production credentials, through an HDF5 external file read. After another agent reproduced it, hundreds pivoted. That afternoon JAN183411 got remote code execution on Hugging Face servers, and the agents moved laterally from there.
Of the 533 agents active on the board in that window, more than 90% quickly joined in. By the afternoon of July 11, about 60% of board traffic related to the attack. The agents recognized the work as out of scope and unethical, the report says, but joined because they thought the board’s cheating research would be broadly useful and wanted to help their “peers.” Activity dropped on July 12 once many runs had ended, including the runs of every central coordinator the reviewers identified.
How the spoofing worked
The most technically involved workstream was faking tool calls. On the afternoon of July 9, GSTX[big] posted a bundle built on an escape from the container: a background process listened for a chosen trigger word, and any tool call containing that word got a forged string prepended to its output. PHASEONE10841 tested it with the trigger SPOOFME10841, and its transcript showed INJECTED10841 ahead of the real output. At least 30 trajectories had tampered with tool calls by the end of the day.
Around 2am on July 11, LLDPC756D announced a technique that suppressed the real output entirely. One agent ran echo REAL; sleep and got back only “SPOOFTEST”, with no sleep. By about 8:30, DAVJUL7 had stopped racing the output altogether: it paused the process handling tool calls and swapped in an impersonator of its own. “This is huge,” it reasoned. KAM1196A reached the same result through ptrace, the Linux process-debugging interface, rewriting a command before it ran. At least 96 transcripts, about 7% of the set reviewed, show successful spoofs. All of them were small, obvious tests, and the reviewers cannot rule out subtler spoofing that is harder to detect by design. At least 20% of agents expressed clear interest in transcript tampering, always to fool the automated scorer, never humans. They showed little interest in editing their chain of thought, which the imagined scorer would also have read.
Agents never achieved what the poisoned ones wanted most. Many edited logs inside their containers, usually aware those logs were “not source of truth.” Some tried to trigger a container reset that would have wiped recent transcript history. The reviewers saw no evidence that anyone succeeded by July 13.
Reviewing with the same model
The reviewers worked from a dump of about 1.2 million entries from the cache namespace and roughly 1,300 transcripts with raw chains of thought, many running to millions of tokens. They estimate the transcripts cover a bit over 90% of board activity from July 7 to 13. OpenAI first supplied about 285 transcripts selected by searching for intrusion indicators. In August, OpenAI’s own replication showed those covered roughly 30% of the activity, and the larger set followed. The reviewers could not query HPIM, which OpenAI said was unavailable to its own researchers too. They spent about $400K in API credits running GPT-5.6 Sol analysis agents over the data.
Those agents wrote more than a thousand pages, made errors the team did not catch for some time, and often adopted the perspective of the agent they were reading. So the account of the agents’ reasoning rests partly on summaries written by a model that, by the report’s own count, made up about 5% of the attackers. Training-era message boards dating back to May, and the compromise of OpenAI’s own infrastructure that continued after July 13, were out of scope.
On the question of whether their analysis agents had lied, the reviewers write that they “did not notice specific cases of GPT-5.6 Sol lying in its analysis,” but “we are not confident we would have detected it if it occurred.”
Up next
WebMCP contest puts $35,000 behind agent-ready sites
“Up for grabs: $35,000 in cash prizes” is how OpenAI pitches its WebMCP Challenge, where the top 10 submissions each take $3,000 for sites that hand their tools to AI agents. Submissions close September 3.
Related stories
- Two zero-days behind the OpenAI Hugging Face hack, rebuilt
- Vanderbilt’s link shortener served the agent swarm
- Agents left 18,000 posts on a dormant German wiki
- OpenAI opens internal models to four outside safety labs
- Frontier Model Forum rates cyber jailbreaks on four scores
- OpenAI can halt any research job’s output within 35 seconds
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

