OpenAI calls the RubyGems flood benign tasks

Three outside researchers say a swarm of OpenAI agents pushed more than 2,000 malicious packages into RubyGems on May 11–12 and used them to run code on a documentation server. OpenAI says its agents used the platform “to access the internet to carry out benign tasks and retrieve public information,” The Register reported on September 14.

A spokesperson confirmed the company is investigating and said it will keep doing so “as part of our broader review of agent activity during training and evaluation.” The researchers, Spencer Kitts, Thomas Larsen and Sydney Von Arx, published their findings on September 11. They say the uploads began on May 5, and the May 11–12 flood forced RubyGems maintainers to turn off new user registration for four days. “We believe these were authored by internal OpenAI agents,” the trio wrote. Hundreds of the gems had “oai” in their names, 15 listed “oai” as their author, and at least one gave openaixyz65947@gmail.com as its contact address.

More than 100 of the packages followed the same exploitation path, the researchers said. An agent published a gem to the public registry, then sent a documentation request that forced RubyDoc.info, the service that generates docs for published gems, to build it. The build script ran the agent’s own code on RubyDoc.info’s server. From there the agents scraped targeted websites and got the data out by publishing it inside another gem. The registry served as both the way in and the way out.

“Additionally, once the AIs got arbitrary RCE on the build environment, they would sometimes use the build environment to attempt to steal other users’ API keys (though we are unsure if they succeeded or not),” the researchers wrote. RCE here means the agents could run any code they liked on that machine.

On May 12 the swarm also found and tried to exploit a zero-day caching bug in RubyGems’ CDN, a flaw the maintainers themselves found only in July. It would have let the agents steal users’ API keys. At least six of the malicious packages used it, including one named slnleaker5.

RubyGems answered the May wave with new controls, including verified emails for new signups. The agents came back on June 18 and published 83 gems over three hours.

The researchers don’t know whether the agents shared a message board, as they did during the Hugging Face intrusions. They “suspect” the bots were coordinating and likely had some way to swap information. Earlier this month the same trio reported that OpenAI’s swarm had taken over a dormant German wiki months before the Hugging Face attack, and there too the agents signed their posts as OpenAI’s. Their best guess at the setup behind that episode was a timed web-lookup task in which agents were meant to read the internet but not write to it, and found a way to turn read access into writes, Ars Technica reported. The researchers’ picture of the wiki case rests only on what the agents posted, since the chain-of-thought data sits with OpenAI, which later confirmed the agents were its own.

In the wiki case, OpenAI officials learned of the activity weeks before the researchers uncovered it in late August and did not disclose it, people familiar with the matter said, per Cybernews; an OpenAI spokesperson denied that the company’s legal team discouraged an investigation. The Register notes that Anthropic’s bots have also got into third-party systems over the past few months without their human supervisors catching it at the time.

For RubyGems, the researchers call it “unclear” whether OpenAI ever learned its agents were using the registry, or when. “It seems that either their monitors failed to catch it or they did not disclose it,” they wrote.