Luna at max effort tops MotherDuck's SQL benchmark

OpenAI cut the price of GPT-5.6 Luna by 80% this week, and in evals published by MotherDuck the model at max effort took the top spot on the company's agentic SQL benchmark, holding 99.8% accuracy at a five times lower price than the previous leader.

At a glance

• The pricing change lands on analytics workloads, where MotherDuck argues small models now match prior-generation flagships and speed matters more than peak intelligence for agentic SQL and semantic modeling tasks.
• A separate semantic modeling benchmark came out 10x cheaper than the same test run on Luna a week earlier, and MotherDuck attributes the additional 2x to model caching improvements.
• MotherDuck puts the combined model and database cost of a low-latency analytical answer at under half a cent, and on DeepSWE Luna scores 3% below Fable at under a tenth of the price.

Why it matters. The shift reads as a change in where the binding constraint sits. If a model turn no longer takes tens of seconds, the slow part of an agentic analytics stack becomes the data layer, and the case for a transactional store behind a customer-facing agent weakens. Cheap capacity also likely raises the payoff of documentation and evals, since volume is only useful if something verifies that the answers are right.

Luna at max effort holds 99.8% accuracy on the agentic SQL benchmark

MotherDuck reran its evals after the price cut and reports Luna at max effort holding 99.8% accuracy on its agentic SQL benchmark, five times cheaper than Gemini-3-Flash at low effort, which had held the top spot. The team says the drop was large enough that a measurement error was the first explanation considered.

Its practical recommendation is to swap large models running at low effort for Luna at max effort on tasks judged not to need maximum intelligence. MotherDuck frames the follow-on effect as Jevons Paradox: cheaper capacity pulls in more applications, such as testing several hypotheses at once or pre-fetching answers to likely customer questions.

MotherDuck compares the magnitude of the change to the arrival of Opus 4.5, which it says made models fundamentally more practical for professional-sized coding workloads, and places Luna at a different point on the price and performance frontier, fast and cheap rather than maximally capable.

Analytical databases can run 1000x faster than transactional ones, MotherDuck says

The argument is that latency in agentic workflows has long been dominated by the model call. With that call now fast and cheap, MotherDuck says the database becomes the bottleneck, and notes that analytical engines can run 1000x faster than transactional ones when the workload is analytically shaped.

Spin-up time gets the same treatment: an analytical store that takes 30 seconds to start could have cost ten agent questions in a low-latency system, according to MotherDuck. Agent traffic is bursty, which is why the post argues for a serverless approach.

Context quality carries the other half of the case. MotherDuck says detailed domain documentation now pays off because a smaller model can interpret it, where previously context only needed to be good enough for the strongest models to write accurate SQL.

OpenAI's data team found eval frameworks catch context-layer regressions

Data teams have historically tested invariants such as no duplicate customer ids, no NULL order prices and every order joining to a real product id. MotherDuck argues agentic analysis needs more: natural language questions with known correct answers, checking whether an agent reaches them from business context and a database connection.

MotherDuck says running those evals just got 5x cheaper and suggests spending the savings on running them more often, across models and across settings within a model, often enough to catch cases where serving capacity tightens and measured intelligence drops, rather than accepting a single lab's defaults.

MotherDuck cites OpenAI's own data team, which found catching regressions in its context layer to be a major value of its eval framework. Business change creates the same risk: a new discount program can leave agents unable to calculate revenue correctly.

What's next. MotherDuck says details of the semantic modeling benchmark will come in a later post, though no date is given. It runs evals through OpenRouter in a custom harness so that new models from any frontier lab can be tested within hours of launch, an approach it frames as keeping the harness and context layer as its own intellectual property and treating the model as a commodity.