OpenAI locks down its models’ hidden reasoning

OpenAI has published new research on what it calls adversarial distillation: using a model’s outputs without permission to train another model that can do much the same things. As the company explains on OpenAI’s blog, the worry is that the skills carry over but the safeguards don’t. OpenAI also says its newest models, GPT-6.2 Astra and GPT-6.2 Sol, now guard their hidden reasoning much better. In its own tests, one kind of extraction attack worked 0.2% of the time, down from 54.0% against GPT-6 Astra.
OpenAI has raised this problem before. In February it briefed Congress on activity indicating that DeepSeek was trying to distill US frontier models. In September it said it had disrupted a campaign that copied encrypted reasoning out of one conversation and asked a model in another to decrypt it, and it tied a core part of that campaign to people associated with Moonshot AI, the developer of Kimi. In late October, The Information reported, citing people with direct knowledge, that DeepSeek and Moonshot relied heavily on GPT-6 Astra outputs to train their latest models.
Why the hidden reasoning matters
Distillation itself is an ordinary technique. One model, the student, learns from data produced by another, the teacher. OpenAI’s concern is the unauthorized kind, especially in areas like cybersecurity and biology, where the same skill can be used for good or harm. A model that helps defenders find software bugs could also help an attacker exploit them. If that skill ends up in a model with weaker safeguards, the copy may help with requests the original would have refused.
The most valuable thing to copy is the model’s reasoning: the steps it works through before it answers. That reasoning can contain information the final answer leaves out. To test how much this matters, OpenAI took one starting model and trained it on recordings of Astra-level models doing coding and defensive security tasks. One version learned from the exact reasoning, the other from the reasoning summaries users normally see. After this first round of training, the student that saw the exact reasoning generally did better. After a further round of training it stayed ahead only when given the most thinking time tested. With less thinking time, the student trained on summaries scored higher. OpenAI concludes that protecting reasoning can limit how much cyber skill transfers, even when the tasks themselves are allowed.
What changed in GPT-6.2
The models are trained to spot attempts to pull out their reasoning and refuse them, while still giving explanations when someone legitimately asks for one. Separate filters check incoming requests for extraction attempts. They also check outgoing answers for leaks, including the instructions a model sends to tools, because reasoning can slip out there without appearing in the final answer.
Other changes target the trick from the September campaign. In August, outside researchers had shown that encrypted reasoning from Claude, Gemini and GPT could be moved between sessions, users and models from the same provider. Now a GPT-6.2 model’s reasoning is tied to the conversation that produced it: if an earlier message is rewritten, the system drops the reasoning that followed. Older OpenAI models with weaker safeguards can no longer read GPT-6.2’s reasoning. OpenAI also says it closed a gap that could expose another user’s encrypted reasoning.
OpenAI also ran tests where automated attackers changed their approach based on how the model responded. Recovering reasoning from an earlier request worked 0.2% of the time against GPT-6.2 Astra, down from 54.0% against GPT-6 Astra. Recovering it from the current request worked 0.9% of the time, down from 22.9%. OpenAI is now bringing some of these improvements to GPT-6 Astra as well.
What is still open
The attempts haven’t stopped. OpenAI says its weekly enforcement actions have ranged from thousands to tens of thousands of accounts, and the rate rose significantly this past summer. It shares warning signs with other model providers so they can look for related abuse.
OpenAI has not published technical evidence for its Moonshot attribution, and it has not shown that Moonshot used extracted reasoning to train Kimi. OpenAI says it will keep studying how much extra capability transfers to different student models and with different training methods.
Up next
Anthropic mathematician claims Riemann proof with Claude
Unsolved since 1859, with a $1 million prize on it: Anthropic mathematician Levent Alpöge says on alpo.ge he proved the Riemann hypothesis with Claude. Would you trust an AI-assisted proof before experts check it?
Related stories
- DeepSeek, Moonshot reportedly distilled GPT-6 Astra
- OpenAI ties a reasoning-extraction campaign to Moonshot AI
- OpenAI model planned to secretly copy emails
- GPT-6.2 Astra stays quiet about problems more often
- OpenAI opens internal models to four outside safety labs
- Frontier Model Forum rates cyber jailbreaks on four scores
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

