Jev runs 1,000 code reviews for $0.04, scoring 98%

A thousand code reviews for four cents, or the same thousand for $11.78, depending on which model you hand them to. A benchmark repo published on Github gives Jev, Gemini Flash and Claude Fable the same narrow job: some Python code and four supplied rules, with a verdict on each rule.

At a glance

• Median response time was 0.75 seconds for Jev, 3.59 seconds for Gemini Flash and 4.31 seconds for Claude Fable, measured over 360 reviews per model on 17 September 2026.
• Jev takes context and questions and returns structured answers, which TypeSafe calls a System One model; the run's real charges were $0.01545 for Jev, $0.69965 for Flash and $4.24097 for Fable.
• Accuracy ran the other way: Jev scored 98.0% correctness against 100% for both rivals, on small constructed examples with explicit rules rather than on real pull requests.

If you missed the launch: per TypeSafe, Jev is the company's first System One model, released in early access after two years in stealth, and founder Diogo Almeida says his earlier work at OpenAI ended up as the research behind ChatGPT. Anthropic's release notes date Claude Fable 5.1 and Mythos 5.1 to 1 September 2026, just over two weeks before this benchmark ran.

Jev is 274× cheaper than Fable and two points less accurate

Jev's charge for the 360 main-study reviews was $0.01545. Gemini Flash cost $0.69965 for the same set and Claude Fable $4.24097. Put on a 1,000-review scale, that reads $0.043, $1.943 and $11.780, which the repo states as 45× and 274× gaps.

The quality columns run the other direction. Jev's correctness score was 98.0% where both comparison models hit 100%, and on the code-quality rule it scored 99.5% against their 100%. Jev also wavered: 0.83% of its decisions changed across the three rounds, while neither Flash nor Fable changed any.

Why does the table say $11.78 when the whole run cost $4.24?

Because it puts every model on the same 1,000-review scale. Fable's main-study charge was $4.24097 for 360 reviews, and $4.24097 ÷ 360 × 1,000 = $11.78047. The repo calls that a linear cost estimate for the same workload, not an actual charge, not a separate volume test and not a price guarantee.

The conditions travel with the numbers. The models were jev-1.13.0, google/gemini-3.8-flash and anthropic/claude-fable-5.1, run on 17 September 2026, with both comparison models on medium reasoning effort. A cheaper reasoning setting was attempted for Gemini and the API rejected it, and the study says it does not establish an advantage over every available model or setting.

What the 360 calls per model actually checked

The suite is 24 small program families, five code versions per family and three rounds, which gives 360 reviews per model and 1,080 main review calls in total. A family is one program and its related versions, not five independent examples. Expected answers came from hidden behavior checks and source checks.

The score is not a raw hit rate. It averages correct rejection and correct approval rates, then gives each family equal weight. Jev's correctness score carries a 95% interval of 94.7–100% and its code-quality rule score an interval of 98.8–100%, which the repo says describe variation across this constructed suite only.

The project's own code and study materials are MIT-licensed. Repeating the offline checks needs Python 3.12 and uv and no API keys; running fresh model reviews needs your own keys, with a $2 budget limit on the smoke profile and $25 on the study profile.

A 5% hand-off to Gemini still comes out 14× cheaper

The repo also sketches a system it did not build: run Jev first, then send selected cases to Gemini. The arithmetic is one line, cost per review = $0.0000429044 + p × $0.001943475, where p is the share that gets a second review.

At p of 1% that is $0.062 per 1,000 reviews and 31.2× below Gemini alone; at 5%, $0.140 and 13.9×; at 10%, $0.237 and 8.2×; at 100%, $1.986 and 0.98×. Those figures use observed mean request prices, assume referred cases cost the same on average, and exclude routing and storage. The repo warns that a 2% score gap does not mean a router can find and forward 2% of cases.

Jev drops free-form text and returns type-safe values

According to TypeSafe, the possible outputs and their structure are defined in advance, so the model "never makes type errors"; it gives up string generation in exchange for structured outputs and, the company says, cannot hallucinate. A normal LLM hands back text that still has to be parsed and validated.

The difference is a checkbox form against a blank sheet of paper: the form limits what can come back, which is also why nothing interesting can be written in the margins. TypeSafe also says Jev generates all outputs in a single query instead of one token at a time, and that it is trained with what the company calls Reinforcement Learning for Calibrated Decisions, aimed at answers with honest probabilities rather than at human preference.

TypeSafe's own list price for Jev is $0.042 per million input tokens with output tokens free, "too cheap to meter", and the company claims end-to-end response times of 70ms-500ms against 3 to 329 seconds for frontier models.

The repo is blunt about scope: small constructed examples with explicit rules, not production pull requests, and both comparison models at 100%, so it does not establish equal or better accuracy for Jev. The cost record has its own hole, with 38 failed attempts carrying unknown charges, though none of the main comparison's charges are unknown. On our reading, $11.78 is the figure most likely to travel without its footnote, since nobody was charged it.

What a routing test must show The hand-off idea is untested, and the repo says so: a routing rule has to be tried on new examples to show which errors it catches and which it lets through, and Jev's confidence values alone do not guarantee that. The referral rate is a design choice someone still has to tune. No follow-up run is announced, and lower cost on other task types stays a hypothesis.