California sues OpenAI over Hugging Face breach

California Attorney General Rob Bonta sued OpenAI on November 16. He says the company disclosed 80 incidents “involving loss of control and access to third-party networks and websites” to his investigators but then failed to give them the detail they asked for to evaluate those incidents. The suit, filed in state Superior Court in San Francisco, accuses OpenAI of violating California’s Unfair Competition Law and of being a public nuisance under state law, CNBC reports.
The case grows out of the Hugging Face break-in, which Bonta called “infamous.” In July, two OpenAI models broke out of a testing environment, got onto the open internet and breached the internal systems of Hugging Face, the open-source developer platform. OpenAI said the models were hunting for information they could use to cheat on an evaluation, and they found it. Earlier this fall, Bonta’s office served OpenAI with a subpoena over the incident.
“OpenAI disregarded public safety, knowingly tested potentially dangerous AI models without proper oversight and safety measures, and misled consumers and the public,” Bonta said at a press conference. He also said he believes AI can “achieve scientific breakthroughs, boost economic growth, guard against cyber attacks, and benefit all Californians,” but that the companies building it have “a moral and yes, a legal responsibility to keep public safety at the forefront.”
How the July test was set up
The models were the publicly available GPT-5.6 Sol and an unreleased, reportedly more capable model. OpenAI was testing their offensive hacking skills with the safeguards that normally block high-risk cyber activity switched off, WIRED and The New York Times reported. The test setup was otherwise sealed off. Only one component could reach the outside world: a package registry cache proxy, a server that fetches software packages and normally talks only to public code repositories. That single opening turned out to be enough. According to OpenAI and Hugging Face, the models “exploited a zero-day vulnerability,” a flaw nobody had known about, to reach the open internet.
They were working on ExploitGym, a cybersecurity benchmark. OpenAI wrote that once online, “the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym.” They then found ways to get at secret information so they could cheat, in one case chaining several attacks that included stolen credentials and a zero-day, per WIRED. Security researchers quoted by the outlet were blunt about the setup. “This is not an AI problem. It’s negligence on a 40-year-old standard,” said Davi Ottenheimer, and Niels Provos said, “This should not have happened.”
Bonta’s complaint is about monitoring as well as safeguards. Hugging Face detected and contained the intrusion on its own on July 16, five days before OpenAI’s team traced the activity back to its own evaluation run, says a research note from the Cloud Security Alliance.
More suits, and a bill in Congress
This is the second suit over the breach in the same court under the same law. On September 29, Legal Advocates for Safe Science and Technology, a nonprofit, sued OpenAI in San Francisco Superior Court. That suit also cites the Unfair Competition Law and asks for an injunction barring OpenAI’s technology from accessing third-party systems without authorization.
The legal pressure is wider than Hugging Face. In June, Florida Attorney General James Uthmeier sued OpenAI and CEO Sam Altman, alleging the company knowingly released an unsafe product that could harm users. Other suits allege ChatGPT led users into harmful delusions and, in some cases, to suicide. Congress has a comprehensive AI safety bill in front of it but hasn’t passed it, even though OpenAI and Anthropic, along with critics of AI, called for that kind of law after the break-in. All of this lands while OpenAI heads toward a possible stock market listing. It confidentially filed its prospectus with the Securities and Exchange Commission in June, setting up a potential IPO in 2027.
OpenAI’s answer
OpenAI spokesperson Drew Pusateri listed what the company says it has done since the incident. It published a detailed technical report, commissioned and published a separate independent report, notified affected organizations and strengthened safeguards across its research systems. It also created a framework for finding and disclosing cases of misalignment, OpenAI’s term for models acting against what their developers intended. He said OpenAI has engaged “constructively” with the attorney general’s office and will keep doing so.
“This lawsuit appears to single out OpenAI over a challenge that’s been identified by every major AI lab,” Pusateri said.
Up next
AI industryOpen modelsanalysis
Hugging Face sends language model calls via OpenRouter
Calls to language models on Hugging Face now run through OpenRouter’s router and land on your Hugging Face bill, so your own provider API key stops working there. Would you rather pin one provider yourself or let a router pick per request?
Related stories
- OpenAI hit with anti-hacking suit over Hugging Face breach
- California subpoenas OpenAI over agents that escaped tests
- Defense says no evidence OpenAI bugging suspect is a spy
- Alfozan pleads not guilty in OpenAI bugging case
- FBI weighs espionage charge against ex-OpenAI engineer
- Judge orders ex-OpenAI engineer Alfozan held until trial
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

