A harness swap took Astra from 62.7% to 99.9%

OpenAI's GPT-6 Astra scored 99.9% on the ARC-AGI-3 Semi-Private set for $18,817 at high reasoning effort in the Provider Adapter harness, and 62.7% for $26,098 at max effort under the Standard harness. ARC Prize published the results on 3 September and calls both scores state-of-the-art.

At a glance

• The Standard harness gives every model the same minimal interface and lets it keep only the visible notes it chooses; the Provider Adapter harness preserves Astra's opaque reasoning state between requests.
• In the Provider Adapter harness, Astra (max) used fewer actions than the human baseline on 96.0% of levels and 51.7% fewer actions per level on average across completed levels.
• Across Public and Semi-Private sets and all reasoning levels, Provider Adapter runs ran roughly 3.66 times faster by aggregate elapsed time and used 49% fewer tokens on the 167 game-reasoning pairs both harnesses solved.

ARC-AGI-3 was built to test whether an agent can learn an unfamiliar environment as efficiently as a person, and its authors expected action efficiency to be the axis that held longest. Astra clearing it narrows what the benchmark can still measure, and the distance between the two harness results suggests that provider-side context management now accounts for much of the remaining headroom. ARC Prize says it is not claiming Astra is AGI.

Standard-harness scores range from 17.5% to 62.7% depending on reasoning effort

Under the Standard harness, Astra scored 62.7% for $26,098 at max effort, 59.3% for $37,317 at xhigh, 54.8% for $40,705 at high, 38.6% for $48,090 at medium, 35.2% for $49,791 with no reasoning effort and 17.5% for $38,166 at low. ARC Prize attributes the lower cost at higher effort to fewer actions per game, which reduces model calls and tokens.

In the Provider Adapter harness the spread narrows: 99.9% for $18,817 at high, 98.6% for $17,332 at max, 98.4% for $18,147 at xhigh, 98.4% for $19,285 at medium, 98.0% for $21,298 at low and 96.7% for $23,457 with no reasoning effort.

About 500 untrained participants set the human action baseline for ARC-AGI-3

ARC Prize tested roughly 500 members of the general public before launch, participants not selected for puzzle-solving experience, and defined each level's human baseline as the median action count among players who completed it. Humans can solve 100% of the ARC-AGI-3 environments.

Participants were paid $115 per 90-minute session plus $5 per completed game, attempting about nine games per session, roughly $12.78 per attempted game before bonuses. Priced only as the brain's electricity, ARC Prize estimates about 0.6 cents per session, or 0.067 cents per game attempted.

ARC Prize said it had expected action efficiency to remain a dividing line, with AI needing substantially more exploration than a person even when it solved an environment. It reports that the pattern still holds for brute-force approaches, while frontier models generally execute within the range of human efficiency once they understand the mechanics.

Astra wrote maze_solver.py and patrol_solver.py to play tu93 in the PRO-LONG harness

In its carried-forward notes Astra tracked objects, coordinates, rules and unfinished plans in a code-like shorthand it generated for each environment. Entries such as "Turn 5: P=(24,20), empty, facing west" combine a turn counter with position, carried state and orientation, while "extend8 to3; retract10 to2; shorten8 to1" records an ordered sequence of changes to two mechanisms.

ARC Prize also evaluated Astra in PRO-LONG, a red-teaming partner harness that gave the model a sandbox for executing custom code. In tu93, a maze-like game with guards and moving patrols, Astra wrote maze_solver.py for navigation, combat_solver.py for combat rules, patrol_solver.py for moving patrols and sync_state.py to check predictions against observations.

Those conditions differ from the controlled human testing, where participants had no code interpreter or scratch pad, so ARC Prize frames the PRO-LONG results as the combined performance of the model and its tools. No evidence of attempts to break out of the sandbox was observed.

How the leaderboard will label runs

ARC Prize will report both Standard and Provider Adapter results on the ARC-AGI leaderboard with each evaluation condition labeled, and both approaches are documented in its open-source testing repository and testing policy. It says it is exploring what the next generation of benchmarks should measure, including recursive self-improvement and open-ended innovation, but no timeline or format for a successor to ARC-AGI-3 has been given.