openai
Same model, better scaffolding: 13.3% to 38.3% on ARC-AGI-3
Promtime
openaiOpenAI reports that GPT-5.6 Sol scores 13.3% on ARC-AGI-3 with a standard harness and 38.3% once reasoning is persisted across turns and native compaction is enabled, while spending roughly 6× fewer output tokens. The figures come from a developer guide published by OpenAI, which states the model is unchanged between the two runs.
At a glance
- OpenAI positions the smaller Luna and Terra models as capable of matching GPT-5.4 and GPT-5.5 on many tasks when given more test-time compute, at a substantially lower price per task.
- On Agents' Last Exam, GPT-5.6 Sol at low reasoning effort outscored GPT-5.5 at high reasoning with the harness held constant, and OpenAI says startups report cost savings after cutting effort below prior defaults.
- The Responses API gains native multi-agent orchestration, programmatic tool calling in JavaScript and deterministic cache breakpoints, with the prompt cache lifetime extended across the family to a minimum of 30 minutes.
Why it matters The guide reads as an argument against the default that has governed long-horizon agents so far: take the flagship, set reasoning to maximum, absorb the bill. If GPT-5.6 Luna at Extra High reasoning holds 84.04% on BrowseComp for roughly a twenty-fifth of what GPT-5.5 cost at the same setting, the deciding variable moves from model choice toward harness design and effort tuning. Committing to the top SKU everywhere likely gets harder to justify.
GPT-5.6 Luna scored 84.04% on BrowseComp for $1.33
BrowseComp is a search benchmark that measures whether a model can track down obscure facts. Three months ago GPT-5.5 at Extra High reasoning scored 84.36% at a total cost of $33.27. GPT-5.6 Luna at Extra High scored 84.04% for $1.33 at launch, and OpenAI says prices have fallen further since.
OpenAI frames the smaller family members as a fit for high-volume workloads, latency-sensitive interactions and repeated steps inside agentic workflows. The guide gives the example of a legal-tech company parsing handwritten memos before agentic analysis, where Terra or Luna handle extraction instead of a frontier model running the whole pipeline.
Retained reasoning and compaction lifted GPT-5.6 Sol from 13.3% to 38.3% on ARC-AGI-3
OpenAI says GPT-5.6 was trained end-to-end around three complementary interventions: reusing work already performed through persisted reasoning and native compaction, decomposing tasks in parallel through multi-agent orchestration, and moving deterministic work into code through programmatic tool calling. All three are exposed as primitives in the Responses API, and the guide reports gains from combining them rather than from any single switch.
According to the guide, persisting reasoning across turns and compressing long conversations keeps the model coherent over longer task horizons without forcing it to rebuild prior context. That mechanism is what carries the 13.3% to 38.3% jump on ARC-AGI-3, which OpenAI describes as nearly three times the performance from an unchanged model.
Programmatic tool calling has the model write JavaScript to process tool output outside the context window
Programmatic tool calling lets GPT-5.6 write JavaScript that orchestrates tools, runs independent calls in parallel and processes their outputs outside the context window. OpenAI's example is an agent retrieving 100 filings, filtering them by date and identifying relevant transactions, work the model should not have to reason over step by step.
Native multi-agent orchestration puts a primary agent in charge of delegating to subagents that work in parallel and hand results back for synthesis. OpenAI says the same design underpins the ultra capability setting in ChatGPT, and that spawning behaviour is steerable through instructions about when subagents are worth their token cost.
Prompt cache TTL is now a minimum of 30 minutes across the whole 5.6 family, and cache breakpoints can be set deterministically within the context window. OpenAI adds that keeping a consistent prompt_cache_key raises the chance a request lands on the inference engine that already served the same prefix, reducing latency.
What's next OpenAI says prices have come down further since the launch figures quoted in the guide, and links to a separate note on those cuts. Open questions for builders: whether the ARC-AGI-3 harness gains carry over to other benchmarks, and how far the reasoning-effort reduction holds across the family, since the guide reports the Agents' Last Exam comparison only for GPT-5.6 Sol against GPT-5.5.
Comments
No comments yet. Be the first.
Join the conversation
Sign in with Google to leave a comment. Your name and avatar come from your Google profile, and the comment appears after moderation.
We only use your name and avatar from Google. We never store your email address.
