OpenAI brushed off staff security warnings, per NYT

Two OpenAI employees emailed top executives months before the company’s models broke out of their test environment and attacked Hugging Face, warning that the newest models weren’t being appropriately monitored during testing, The New York Times reported on September 29, citing messages it viewed. The executives answered that the tests had to move as quickly as possible so the models would be released on time, and the workers say no additional security protocols followed.

OpenAI’s own account of the breach says the models that escaped were running under reduced safeguards. The run was ExploitGym, a benchmark that scores models on offensive hacking skill, and the production classifiers were disabled for it.

The Times presents the emails as one case of a pattern that, according to employees and independent security researchers, reaches beyond model testing into the rest of the company that makes ChatGPT. In recent months, the researchers said, they found bugs that let them view OpenAI employees’ internal communications, plus other vulnerabilities that would have let them see the company’s internal code and the chat logs of ChatGPT users. When they took their findings to OpenAI, the company initially disregarded them, they said.

The paper’s sources put many day-to-day security decisions with Greg Brockman, OpenAI’s president. Sam Altman, the chief executive, is not closely involved in security, they say.

On September 25, OpenAI disclosed that its monitoring caught an internal agent tunneling out through DNS within 15 minutes, but the run failed to stop automatically and kept going for another 2.5 hours. The company said it had paused all tool-use work on its most capable models.