Jev answers in 195 ms, OpenAI Decisions in 208 ms

TypeSafe’s Jev finished 95% of its requests within 238 ms in a six-city latency test run October 7-20. OpenAI’s Decisions API needed 707 ms to clear the same bar, and Cloudflare’s Clef needed 997 ms. On a typical request the three were much closer: across 990 requests, Latencyradar measured median full answers at 195 ms for Jev, 208 ms for OpenAI and 344 ms for Clef.

On a typical request Jev leads OpenAI by only 13 ms. At the slow end the gap is 469 ms. All three are decision APIs, which return a structured answer (a yes/no probability, a pick from fixed options or a score) instead of free text. Each one got the same yes/no question, “The customer is explicitly asking for a refund”, about the same two-sentence support message, five times per city per day. None of them stream, so the clock runs until the whole JSON answer arrives over a new connection. Jev and Clef accept the same request format. OpenAI got the question as a predicate in its own format.

Who won, day by day

Jev was measured for longer: 390 requests over 13 daily runs, against 300 requests over 10 runs each for OpenAI and Clef, which Latencyradar added later. On the ten days when all three were tested, October 11 to 20, OpenAI had the lower daily median on six. It opened at 142 ms against Jev’s 207 ms on the first of those days and was still ahead on October 18, at 182 ms against 183 ms. Jev won on October 14 (174 ms against 236 ms), October 16, October 19 and October 20 (183 ms against 235 ms).

Over those ten days OpenAI’s daily median moved from 142 ms on the first day to 235 ms on the last, with a peak of 265 ms on October 16. Jev’s daily medians stayed between 174 and 230 ms, and Clef’s between 284 and 472 ms. For none of the three was there a day when every city slowed down together.

OpenAI’s cold start

Most of OpenAI’s tail probably comes from one pattern in Latencyradar’s half-hourly status checks: the first request after a quiet spell is the slow one. It took about a second in Amsterdam and nearly two seconds in Mumbai. The two requests right after it took 300 to 400 ms in Amsterdam and 150 to 300 ms in Mumbai. The benchmark’s advice to existing OpenAI customers follows from that: the Decisions API is “nearly as fast on a typical request”, but the first call after idle time costs extra.

OpenAI quoted 150 milliseconds when it presented the Decisions API in September. In this test its best city was San Francisco, at 120 ms typical, and its daily medians ran from 142 to 265 ms.

Does the city matter?

It matters a lot. Mumbai was the slowest location for all three: 287 ms typical for Jev, 312 ms for OpenAI and 647 ms for Clef. Jev and OpenAI were fastest from San Francisco, at 99 ms and 120 ms. Clef’s best city was Montreal, at 248 ms.

Jev had the lowest typical time in five of the six cities. Amsterdam was the exception, where OpenAI answered in 213 ms against Jev’s 229 ms. Even there OpenAI’s p95 was 953 ms, against 274 ms for Jev. City by city, Jev’s p95 ranged from 140 ms in San Francisco to 342 ms in Mumbai. OpenAI’s ran from 310 ms in Tokyo to 1,235 ms in Mumbai. Clef’s ran from 516 ms in San Francisco to 1,509 ms in Singapore, and Clef was the slowest of the three in every city.

TypeSafe gives its own range of 70-500 ms end to end. Every Jev city figure here, typical or p95, falls inside it. Jev does not generate text. A request carries a state and a set of questions, and the API evaluates every question on that state in parallel, in one call. TypeSafe says this replaces the usual token-by-token sampling. That design is a likely reason Jev’s slow requests stay so close to its typical ones.

Price, inputs, weights

Input prices, from each provider’s pages as checked on October 20, are $0.042 per 1M tokens for Jev, $0.10 for OpenAI and $0.24 for Clef. That puts OpenAI at about 2.4 times Jev’s rate. OpenAI and Cloudflare don’t charge for output tokens, and Jev’s output-token price has not been published. All three handle yes/no, choice and score questions. OpenAI accepts image input and Clef accepts up to four images, while Jev currently takes only text.

Clef is the only one of the three with open weights, released under Apache-2.0, and it was the 27B clef model that was measured, not clef-flash. It ran on Cloudflare’s free Workers AI allowance. Two limits apply to every figure here. The page measures speed only, not whether the refund answers were right. And these are times for whole answers, not first tokens, so they shouldn’t be set against numbers from streaming chat APIs.

Jev was measured as jev-1.13.0, pinned to that version. OpenAI’s Decisions API is still a public beta, and its only model, gpt-6-luna, has no dated version, so its results can change without notice.