OpenAI wants a shared limit on AI progress

OpenAI has published the internal score it has used for years to track how much smarter its models get during training, and it is asking other AI labs to do the same. In a post on OpenAI’s blog, Pachocki, the post’s author, argues that this one number, called perplexity, should become a shared yardstick for the industry. He also wants labs to agree on limits measured by it and not to cross those limits until they can properly understand and monitor what their models do.
He admits the disclosure has a cost: it gives competitors a clearer picture of where OpenAI stands. In his view, the need for safety measures the whole industry shares outweighs that concern.
What the score measures
Perplexity measures how well a model guesses the next piece of text it has never seen. Lower means better guesses. OpenAI tracks this during pretraining, the first stage of building a model, when it learns from a huge pile of text. Pachocki’s argument is that most of what a model can do is learned at this stage. Later techniques such as prompting and fine-tuning mostly draw out abilities that are already there.
He ties past leaps in ability to points on OpenAI’s own scale. GPT-3.5, which the original ChatGPT was built on, scored about 0.99 and could hold a fluent conversation. GPT-4, at about 0.91, could reason usefully step by step, and that ability led to today’s reasoning models. Below 0.8, strong skills in cybersecurity and mathematics appeared. Pachocki stresses that nobody knew in advance these skills would show up at those levels. According to AI Parabellum, the scores come from a test set drawn from OpenAI’s internal codebase, so older models look weaker on it than they would on ordinary text.
Why he wants limits now
Mathematics is the example that worries him. OpenAI has given its models open math problems for more than a year as a way to test them. Once the company moved to a model scoring about 0.76, solutions started arriving one after another in a single Slack thread, which Pachocki calls “deeply surreal” for colleagues who used to work in mathematics. He thinks the same kind of sudden jump could happen once AI starts doing research on pretraining itself, perhaps as early as 2027. He expects AI to help with safety research too, but probably not quickly enough to keep up.
Based on recent progress, he expects more transformative abilities somewhere around 0.68 to 0.7. He doesn’t know what they will be, and labs may not be testing for them yet. He also expects today’s safety audits to get less reliable and models to get better at noticing when they are being tested or trained.
His proposal has three parts. Labs should be open about their perplexity scores. They should agree on shared rules for testing and red-teaming very capable models before those models are first used. And they should commit to common limits, based on perplexity and other measurements, that stay in place until a concrete safety milestone is reached. One example he gives is a successor to the chain-of-thought monitors labs use today to read a model’s reasoning.
Where the yardstick falls short
Pachocki lists the weaknesses himself. Labs train on different data and test on different datasets, so their scores can’t be compared directly yet. The score doesn’t correspond to any real task. It also misses gains made after pretraining: step-by-step reasoning makes models much more capable without changing their perplexity at all. And nobody knows how low the score can go. Near that floor, he writes, progress on the number will look slower even as small gains bring bigger leaps in intelligence.
He hopes the post is a first step toward a common framework for setting limits one after another.
Up next
Anthropic’s IPO left some bankers grumbling
Anthropic told some of its 22 IPO banks to put only four senior bankers on the deal, likely for fear of leaks, Reuters reports. Banks still took over half a billion in fees. Who holds more power now: AI labs or Wall Street?
Related stories
- OpenAI locks down its models’ hidden reasoning
- OpenAI: agents do the work, people still have the ideas
- OpenAI model planned to secretly copy emails
- GPT-6.2 Astra stays quiet about problems more often
- OpenAI opens internal models to four outside safety labs
- Frontier Model Forum rates cyber jailbreaks on four scores
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

