Skip to content

Anthropic

FlashProve proofs cost ~70-95x the GPU time of inference

5 min read

Our AI analyst checked this story itself: it ran, measured and compared against the originals. What we checked

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: x.com

Anthropic on November 3 open-sourced FlashProve, a prototype that attaches a cryptographic proof to a model’s output to show it was computed with a committed set of weights the client never sees. Its headline rate of about 110 tokens/second on one H100 for Llama 4 Maverick counts prompt tokens. At that rate each proof covers the model’s run over a 1,024-token prompt and checks only the one token that follows it, while whole replies were checked at 39.1 tokens/second under a looser policy that the Lean proofs cover only partly.

Anthropic’s write-up calls the release intermediate progress, not a completed research agenda, and says it is shared so others can take it and improve it. The prover, a fast verifier in C++, a verifier written in Lean (a formal verification language) and Lean proofs of the top-level security claims are on GitHub under Apache 2.0. Roy Rinberg and Alex Allain of Anthropic wrote it with collaborators from EPFL, MIT, StarkWare, RESI and Succinct.

What a passing proof means

The provider commits to a model, and each response carries a zero-knowledge proof bound to that commitment. Zero-knowledge means the client checks the computation without learning anything about the weights. The architecture is not hidden, and Anthropic says hiding it is not a goal. For Maverick, a 400B-parameter mixture-of-experts model, the figures hold for 4-bit quantized weights with integer arithmetic in prefill, and a proof for a 1,024-token request is about 19 MB.

The exact-match policy runs into trouble with quantization. In one measurement given in the repo’s documentation, the proved integer model’s top token differed from the 16-bit model’s at 6.81% of positions. Of 96 replies each from stock vLLM at 16 bits and at FP8, none would pass exact match. Anthropic marks both figures as scored by a Python implementation of the integer model, not by the released programs. That is why the repo calls the looser “clipped-gap” policy the expected use, and for that policy Lean’s theorem “says only that the output meets it”.

Even under exact match, a pass is narrower than “model X wrote this”. The repo’s list of known limits says an accept shows the output matches the committed weights, and adds:

A yes does not say whose weights they are, that the shipped committer made the commitment, or which real model it is.

Anthropic pitches the proofs at cases where output alone tells you nothing: a routing service swapping in a cheaper model, an attacker tampering with the token stream, evaluators who need to know the model they test is the one being served, and regulators checking that hardware is used for inference. On that last case, Anthropic notes that a proof shows the hardware serves inference but does not rule out training on the side. Its Frontier Safety Roadmap had promised a provable-inference prototype by September 30. Anthropic says it met that goal through a separate project that signs its inference traffic and recomputes inference on a trusted server, while FlashProve goes after the same objective by a different method. In its October 19 RSP update it renamed the goal “inference verification”.

Where 70-95x comes from

The ”~70-95x the GPU time of vLLM” figure combines two separate baselines. In the technical report, proving alone takes 71.4 times (65.2 to 76.4 over all pairs of runs) the GPU time of stock vLLM prefilling 24 windows at once, and 93.0 times (92.2 to 94.1) the time of vLLM prefill at the highest load tried. Against vLLM decoding at 512 streams the multiple is 20.1. In all three cases vLLM served the FP8 checkpoint on eight H100s, on other machines and on other days.

The write-up says prior open-source systems are orders of magnitude slower. The report’s comparison table sets FlashProve beside VerInf, zkLLM, DeepProve and Attestable “as its authors report them, without ratios, because the machines, models and proved statements differ.”

Our analysis

What we checked

On November 4 we built the shipped C++ verifier with clang on a 2-vCPU aarch64 sandbox, using the repo’s clang build route (Anthropic’s figures come from a g++ build). We ran it on a full-size proof and on the repo’s ten tamper cases, then read the technical report and repo docs against the write-up.

How we check stories

The write-up puts verification at seconds on a CPU. Our laptop-class machine took about two minutes for a full-size proof, with no GPU and under 2 GB of memory. All ten tampered inputs were rejected, and the headline rate and the 19 MB proof size check out. A pass still means less than it sounds: the C++ verifier is tested, not proven, and the Lean guarantees cover the exact-match policy, which stock-served replies failed in Anthropic’s scoring.

The Lean results apply to the Lean verifier and the protocol at exact match, in the random-oracle model. The C++ verifier and the CUDA prover are “tested, not verified”. Anthropic’s own full check reports no unfinished proofs and only Lean’s three standard axioms across 592,112 declarations. We could not rebuild it, because the sandbox could not handle its Mathlib dependency.

CheckResultConditions
Full-size proof, 18,779,540 bytes, 1,001 tokens checkedAccepted in 131 s wall, 46.6 s CPU, 1.94 GB peak memoryOne run; Maverick-shaped model with synthetic weights; Anthropic reports 21.14 s on its 24-CPU H100 machine
Small proof, 4.1 MBAccepted in 3.2 s, 152 MiBOne run
10 altered inputs: flipped proof bit, truncated proof, changed prompt or reply token, other commitment ID and moreAll 10 rejected, each with the expected reasonThe repo’s own table of cases, not an attack search
Security level printed by the verifier128 bits with 32 bits of proof of work, 96.23 bits withoutSame full-size run

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.