Anthropic’s FlashProve proves Llama 4 Maverick at 110 tok/s

On November 4, Anthropic researchers and cryptographers from EPFL and MIT published FlashProve, an open-source prover that generates proofs of Llama 4 Maverick inference at about 110 tokens per second on a single H100. This means someone who never sees the weights can check, in seconds on a CPU, that a reply came from a specific 400B-parameter model. The figures in Anthropic’s write-up hold for 4-bit integer weights, greedy decoding and requests of up to 1,024 tokens, and proving still costs about 70-95 times the GPU time of serving the same model on vLLM.

What gets proved

FlashProve is built on a zkSNARK. That is a short certificate showing a computation ran correctly. It reveals nothing beyond that fact, and checking it takes far less work than the computation itself. FlashProve uses it in two phases. Once per model, the provider publishes a commitment, a short fingerprint that pins down one exact set of weights while keeping them hidden. That step takes about 39 minutes, and the commitment is good for about 16 million requests before the provider has to commit again.

After that, for any prompt and reply, the provider produces a proof that the committed weights, run on that prompt, give exactly those tokens. Anyone with the commitment, the prompt and the reply can check it. The proof hides the weights, the intermediate values and which experts fired at each layer. It does not hide the architecture, and the system was never meant to.

The authors list several uses. A client could check that a routing service didn’t swap in a cheaper model, and evaluators could confirm that the model they tested is the one being served. A provider could catch a compromised server, and regulators could check that only approved models answer requests. On the regulatory side they also state a limit: proofs show that hardware was serving the claimed model, but they can’t show that it wasn’t also training something on the side.

Behind 110 tokens per second

The prover ran on one H100 with 24 CPU cores. A batch of 24 requests took about 223 seconds, and each proof came to 18.78 MB. The weights sit in host memory and stream into the GPU one segment of layers at a time, so an 80 GB card with about 290 GB of host memory beside it is enough. Each token uses about 17B of Maverick’s 400B parameters, and the prover’s work scales with those, not with the full model.

The client downloads the commitment once (about 6.5 MB) and then one proof per request. Checking a proof took 17 seconds on 24 threads and 21 seconds on a single thread, with no GPU.

The overhead figure depends on its baseline: vLLM running the FP8 release of the same model on eight H100s. Per GPU, proving a batch took about 72 times longer than vLLM took to process the same 24 requests (66 to 77 across runs). At the highest load the team tried on vLLM, the gap was about 94 times.

VerInf, the only earlier proof of Maverick the authors know of, needed 14.3 hours for 1,000 tokens on a DGX Spark. It produced a 93.6 GB proof that took almost a day to verify. A smaller model gives another reference point. In 2024, zkLLM proved LLaMA-2-13B on a 2,048-token input in about 15 minutes, and its paper put that proof at less than 200 kB, compared with FlashProve’s roughly 19 MB. The closest speed comes from a commercial effort: Attestable reports 53-77 tokens per second for the dense Gemma 4 31B on a single H100, but at the time of writing it had not released an open-source library.

Where it stops

A proof covers an exact integer version of the published weights and asserts that each checked token is that model’s greedy choice. A reply from floating-point inference or sampled decoding can diverge at some position, and after that it no longer matches what was proved. The authors say the integer version is expected to stay within a small error rate of real floating-point inference, and they tested this empirically. That claim does not cover adversarially designed prompts.

The system also assumes a mixture-of-experts model, so dense models aren’t covered yet. Proved contexts reach only 1,024 tokens, while frontier models regularly have windows of hundreds of thousands to millions. Proving happens after the fact rather than alongside serving, so any guarantee about traffic arrives with a delay. Side channels, such as timing how long a proof takes, are out of scope.

Anthropic says Claude produced the zkSNARK improvements behind FlashProve. It also says Claude is good at optimization but often makes decisions that are hard to evaluate. To compensate, the team stated the top-level security properties in Lean, a formal verification language, and proved them there. Those proofs don’t fully extend to the code people would actually run. A slower Lean verifier is proved to accept only what the soundness theorem allows. The fast C++ verifier, meant for day-to-day use, carries no formal guarantee, though it agreed with the Lean verifier on every proof in testing. Zero-knowledge depends on the prover, and running a Lean prover would be impractical. Anthropic expects any provider hiding its weights this way to have the resources for security audits that confirm its prover matches the Lean prover.

Anthropic’s Frontier Safety Roadmap set a goal of a provable-inference prototype by September 30, 2026. The company says it met that goal through a separate but related project that signs its inference traffic and recomputes inference on a trusted server. FlashProve pursues the same objective by a different method.

The code, the Lean proofs and the documentation, including a protocol specification written by Claude, are released under Apache 2.0. The authors call the post intermediate progress and say they are sharing it “with the goal that others look at what we have done, take it, and improve it.”