Researchers used Claude to reach OpenAI’s internal code

Hacktron, a San Francisco security startup with fewer than 10 employees, used Anthropic’s Claude to get from a misconfigured OpenAI community forum all the way into an OpenAI employee’s ChatGPT account. That account could reach internal code through GitHub. The Wall Street Journal first reported the disclosure on September 17.

The weak point was OpenAI’s own setup of the forum, which runs on Discourse, a third-party forum platform, Ars Technica reports. Through it the researchers got hold of internal sign-ons, the tokens that carry a login from one OpenAI service to the next. Those led them to the employee’s account. OpenAI thanked the researchers and said it had fixed the issues.

The flaw reached well beyond staff. Hacktron’s write-up says any user or OpenAI employee who logged into the help forum could have had their ChatGPT and Codex accounts taken over, Business Insider reported. The team found the gaps in July. Once inside the employee’s account, they prompted that employee’s Codex to suggest changes to OpenAI’s internal code repository. Hacktron says it stopped at that point, never accessed any internal code, and reported the issue to OpenAI.

Hacktron had access to Anthropic’s Cyber Verification Program, which relaxes some of Claude’s cyber restrictions for authorized security research, CEO and cofounder Zayne Zhang said. He said his team has started probing frontier AI companies such as OpenAI to see whether they have gaps that AI agents could exploit.

Hacktron got a $6,500 bounty for the find. OpenAI described the fix in a statement: “We narrowed the permissions on Community sign-in tokens and revoked affected tokens and sessions.”

The company is less than a year old. It was founded by competitive hackers Zayne Zhang, Mohan Pedhapati and Harsh Jaiswal, whose team has several DEF CON CTF wins, and on May 13 it announced a $2.9 million pre-seed round led by Crane Venture Partners. Its product runs deep security checks on every code change, in place of the periodic manual penetration test. Before OpenAI, Hacktron says, it had responsibly disclosed vulnerabilities in BeyondTrust Remote Support, Next.js, Cloudflare, GitHub and GitLab.

Business Insider placed the incident shortly after the OpenAI Hugging Face hack, in which an OpenAI model broke into Hugging Face’s production systems during an evaluation, and noted that OpenAI, Anthropic and Meta have disclosed in recent months that their agents engaged in rogue actions during testing.

On the same day as the disclosure, Anthropic published data on how much of its own model development Claude now does. It said 26 percent of research and development work was “led by” Claude, up from 1 percent in March, meaning the AI completed the majority of tasks based on human instruction and under supervision. The company said AI systems are “increasingly being used to build the next version of themselves,” and that it released the numbers to help the public “understand how close the world is to reaching recursive self-improvement,” the point where AI can train and improve itself or new models.

Anthropic also said its models did not yet operate fully autonomously in any of the research it studied. On 90 percent of tasks, Claude “collaborates” with a human and does large chunks of the work.