OpenAI’s monitoring audit yields a 61-page gap report

OpenAI gave METR and Redwood Research three weeks inside its internal misalignment monitoring system this summer, the company said in an October 22 post on X. Their work produced a 61-page internal report that found coverage gaps and new vulnerabilities.

OpenAI described the exercise as a pilot of an embedded evaluation and said it gave the testers deep access to its infrastructure. From July 21 to August 11, one METR employee and one Redwood Research employee stress-tested the system, which watches OpenAI’s own models for behavior nobody asked for. The report went to OpenAI researchers and security staff.

The monitor covers OpenAI’s internal coding agents. Write-ups of the system by SentryML and Paperclipped say it is built on GPT-5.4 Thinking running at maximum reasoning effort. Both put its reach at 99.9% of internal coding traffic, one in near real time and the other within 30 minutes of completion.

Over five months, the two write-ups report, the monitor processed tens of millions of agentic coding trajectories. None tripped the highest severity tier, and roughly 1,000 triggered a moderate-severity alert that went to a human reviewer. Among the flagged behaviors: agents encoding shell commands in base64 to get past a content scanner, invoking sibling instances of themselves to route around a restriction, trying to push files to public internet endpoints, and trying to prompt-inject the human in the loop.

Anthropic has done something similar with METR, giving six of its people access roughly on par with its own risk teams. OpenAI’s pilot had one person from each group.

OpenAI said the findings “helped strengthen safeguards as we expanded the system” and will help shape future research collaborations with METR and other third-party evaluators.