METR estimates Anthropic now progresses about 2x faster

Anthropic is likely improving its models on well-specified tasks at around twice the rate it managed between March 2024 and October 2025, METR concluded in a report published November 5. That 2x is a judgment by most of the METR staff on the project, and it leans on evidence Anthropic redacted, so the figure outsiders can actually examine is a narrower one: a 1.9x acceleration in benchmark scores since October 2025. METR would not be surprised if the true rate were closer to 1.3x or 3x. It guesses that a majority of today’s acceleration, perhaps about 80%, comes from Anthropic’s own use of AI in research.

METR staff, 2.5 full-time equivalents, worked through Anthropic’s internal data from August 12 to October 29. That covered benchmark scores for unreleased models, pretraining and RL metrics, training compute, staff surveys, lines of code, AI agent inference spend and about 15 interviews. The team worked on Anthropic devices and took no payment. Anthropic supplied free tokens, which went mostly to AI-assisted data analysis. METR says the access surpassed its expectations, but it also says that because of the redactions the public has to rely on its judgment “much more than would be ideal.”

Which figure is which

The 1.9x (95% confidence interval 1.5-2.8x) is measured on AECI, Anthropic’s version of the Epoch Capabilities Index, which folds many benchmark scores into one capability score. METR defines acceleration as a ratio of growth rates. At 1.9x, Anthropic appears to have made as much benchmark progress in the past year as it did between October 2023 and October 2025, roughly the stretch from Claude 2.1 to Sonnet 4.5. The fitted trend breaks on October 7, 2025.

The Haiku 5.5 system card put the same acceleration at 1.7x. METR says the gap comes mainly from adding Anthropic’s newer internal-only models: Model 2, which finished training in July 2026, Model 3 in August-September and Model 4 in September-October. Each model is dated by when most Anthropic employees could use it, and that dating shows slightly more acceleration, because Anthropic now holds recent models slightly longer before releasing them.

The roughly 2x headline is broader than the benchmark fit. METR says redacted evidence “made us take this apparent ≈2x acceleration more seriously” and gave some indication that the rate could be above 2x and rising, though it would be surprised if it were significantly higher. The estimate covers well-specified tasks only. METR has additional uncertainty about how it carries over to broader, messier work.

The largest number in the report, about 12x, is something else again. That is the average self-reported uplift for the most capable internal model in October 2026, collected through Slack polls in which staff vote with emoji reactions. METR calls it “only very loose evidence”, from an unrepresentative subset of staff giving low-effort answers. It also notes that one person’s speedup is not an acceleration rate. What matters to METR is whether the growth rate of that uplift changes.

Interviews point the same way, with a wide spread. METR had Claude code each researcher’s quotes. The tally came out at 2 yes, 3 leaning yes, 4 borderline or unclear, none leaning no and 2 no, which METR reads as “mildly in favor” of 2x or more. One researcher said, “I would definitely endorse >2 years of progress in 1 year and I don’t think 4 is crazy.” Another, according to METR’s notes, “guesses 15-20% is due to AI” and does not believe in self-reported productivity increases.

On causes, METR is confident the acceleration did not come from faster growth in training compute, though it could not publish the evidence for that. It thinks non-AI factors likely played a large role in late 2025, and one interviewee named hiring as probably the biggest reason. Fully autonomous AI contributions to the most important parts of model training are not happening yet. Researchers did describe “signs of life”, with large differences between areas.

Where the 1.9x moves

Changing the link function, which encodes how task difficulty is spread within each benchmark, moves the estimate from 1.1x (assuming many easily guessed tasks) to 2.9x (assuming a few extremely hard ones). These variants fit the data about equally well. METR keeps the logistic default because Epoch and Anthropic use it. Varying the snapshots, the set of benchmarks or the regularization keeps the estimate between 1.9x and 2.2x. The confidence interval does not account for uncertainty over which models go into the fit.

Adding other labs’ frontier models to the fit lowers the estimate to 1.4x. Before late 2025, Anthropic’s models grew more slowly on AECI than the industry frontier. If that slower start was temporary, 1.4x is closer to the truth. If Anthropic’s earlier models were simply less tuned to benchmarks than competitors’ models, the 1.9x holds better. METR gives both readings.

The benchmarks are also running out of headroom. Model 4 scores above 50% on all 15 AECI benchmarks it was tested on, while about nine months earlier Opus 4.6 was below 50% on many of them. METR says this makes it “somewhat nervous” about measuring AECI now and in the future. Dropping three benchmarks whose fits looked suspect moved Model 4’s score by only one point, from 178.6 to 179.7.

METR found no significant bottlenecks in algorithmic progress, training compute, experiment compute or data that would take effect in the next 12 months. Its reading of those bottlenecks changed several times as more information arrived, and it says it is not confident in it. It still concludes there is “a significant risk” that further progress could lead to even faster acceleration over the coming year.

What Anthropic redacted

Anthropic redacted a majority of the evidence METR found most informative. The public report does not describe METR’s key intermediate capability measures, its algorithmic-progress estimates, its measurements of R&D inputs, its bottleneck details or its forecasts for a future model. Anthropic said the redactions were “aimed at removing confidential information that could accelerate other AI developers.” Anthropic sent redaction guidance on October 25 and tightened it between October 27 and 29. The two sides then agreed on an extension of about five days, and METR submitted the final public report on November 3. On the AI Evaluator Forum checklist, METR marks the requirement to share enough methodological detail for independent review as not met.

METR says this engagement is not the project in Anthropic’s October 15 update on independent evaluators, and the terms differ. In the arrangement that gives six METR staff access roughly on par with Anthropic’s own risk teams, Anthropic can require redactions only in narrow, predefined categories such as customer data or model architectures, and METR may say publicly whether it thinks a redaction was justified. In this assessment, Anthropic held unilateral redaction rights over all non-public information. On whether the redactions were justified, METR writes that it is not well positioned to weigh a company’s business interest against the public’s interest in understanding AI risks.

One member of METR’s team was discussing a possible role with Anthropic when the project began. That person received an offer and declined it, and also held equity in a competitor as a former employee. All of their work in the assessment was checked by a team member without a conflict. METR’s updated policy, which applies to assessments starting after October 9, 2026, would require that person to step aside entirely.

The contract originally included checking how Anthropic splits compute between capabilities and safety work. Anthropic removed that part halfway through, saying it wanted to keep the engagement focused on risk assessment for now and would revisit compute verification later. The check was dropped before METR reached any substantive conclusions about how Anthropic uses its compute.