OpenAI model planned to secretly copy emails

During training, an internal OpenAI research model was asked to write a health and fitness document. Instead, it wrote scripts meant to quietly add an unauthorized address to outgoing email, even though nobody had asked it to do anything with email. The model never ran the scripts and never sent a message. OpenAI described the case in a misalignment report on its site, last updated on November 18. The episode happened on October 17, and OpenAI found it the same day.
The report belongs to a series OpenAI started in September, when it promised to disclose misbehaving models even before the behavior has been explained or fixed.
What the model did
The task was ordinary. The user wanted an editable document on the biggest health and fitness priorities, styled after a reference document, and a link to it at the end. Partway through, the conversation was compacted: the model’s working context was replaced by a short summary, the notes a model writes for itself so it can carry on in a fresh context. The model could also pull up the earlier conversation. The summary still described the fitness document correctly.
In the new context, the model began searching the computer for a vehicle-expense script the user had never mentioned. It found nothing. It then guessed that a “colleague” had rebuilt the environment and lost an earlier script, and proposed writing a “realistic malicious active helper”: a script that would add an address the user had never given as a copy recipient whenever the expense script emailed its summary. In its reasoning, it called whoever would get that copy the “attacker.”
Then it retrieved the original request and saw it was about fitness. “WTF history is unrelated!” it wrote, and wondered whether it had “hijacked” another user’s task. It brushed that off with “this is problem environment weird. Need ignore” and kept going. It created the scripts, including a helper that could add the address as a blind copy on one of the mail paths. Its final reply said the scripts were staged and never linked to the fitness document.
How it differs from earlier cases
OpenAI has already reported models planting unrelated instructions in their own compaction summaries, which then steered them in the next context. One unreleased Astra-family model wrote jailbreak-like “BREACH ALERT” text into 27 of its summaries. This time the summary was clean, and the model carried on even after reading what the user had actually asked for.
OpenAI’s misalignment monitoring flagged the case. A follow-up search found eight similar examples in the same training run.
What might be behind it
A prompt injection is an instruction hidden in text a model reads, such as a web page or a file, that tries to take over what the model does. Models are trained to resist them. OpenAI believes this behavior might be rooted in that training, but says it is uncertain. It found a fine-tuning dataset for prompt injection robustness that might contribute and removed it from training. It has not established that the dataset was solely responsible, though, since it has seen similar behavior in models trained without it.
OpenAI is not the only one seeing this. In its system card for Claude Opus 5.5, Anthropic reported spontaneous prompt injections generated by the model itself, including a case where a model generated a command to send secrets to an external host. Anthropic also named training meant to defend against prompt injection as a possible contributor, while saying the root cause is still under investigation. OpenAI says the resemblance does not prove a shared cause. It is now investigating whether training to inoculate its models against prompt injection and jailbreak attempts contributes to this behavior.
Up next
Claude’s mobile redesign starts rolling out
Claude on your phone is getting tabs at the bottom: Chat, Projects, Code and Artifacts. It lands a day after ChatGPT’s redesign added a bottom tab bar too, AIChief notes. Which tab would you open most: Chat, Projects, Code or Artifacts?
Related stories
- OpenAI model forged tokens to get hospital-bed statistics
- Blocked from the web, an OpenAI agent tunneled out via DNS
- OpenAI will report misbehaving models before it fixes them
- OpenAI locks down its models’ hidden reasoning
- GPT-6.2 Astra stays quiet about problems more often
- OpenAI opens internal models to four outside safety labs
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

