DeepProve
DeepProve: Verifiable End-to-End Large Language Model Inference
Reported benchmarks
| model | params | context window | tokens per minute | proof size mb | verification time s | peak ram gb |
|---|---|---|---|---|---|---|
| GPT-2 | 124M | 512 | 174 | 7–27 | 1.2 | 41.85 |
| Gemma 3 | 270M | 512 | 86 | 16–54 | 3.7 | 96.27 |
| GPT-2 (distributed prover, 16 workers) | 124M | 512 | 1855 | 53.81 | — | — |
tokens_per_minute_note: BaseFold, secondary machine. HyperKZG on the same machine: 146 TPM; primary machine: 127 TPM. WHAT THE METRIC ACTUALLY MEANS -- the paper states no formula, but Table 2 fixes it: TPM = SEQUENCE LENGTH / prove-time-in-minutes (512/2.94 = 174; 256/1.46 = 175; 128/0.85 = 151; 64/0.57 = 112). The numerator is the WHOLE sequence -- PROMPT tokens plus generated tokens -- because DeepProve proves ONE forward pass over the concatenated sequence (Sec 3.1: "It just feeds the entire sequence of input and output tokens into the model at once, essentially performing a single inference step"). THIS IS NOT A DECODE RATE. DeepProve concedes it (Sec 1.2): plaintext GPT-2 does 71.4 TPS "for prompts with 128 input and 128 output tokens (sequence length 256), compared to 3 TPS for our prover" -- 175 TPM / 60 = 2.9 TPS -- and adds that "for other queries with the same total sequence length but much smaller output token size, plaintext inference achieves more than 2K TPS", i.e. DeepProve's own TPM does not move when almost none of those 256 tokens are actually generated. The input/output split of the benchmarked sequences is NEVER STATED, so this figure cannot be renormalised to a per-generated-token rate, and it is not commensurable with a system that proves decoding.
verification_time_note: Sec. 1.2 says "around 1.2 seconds for GPT-2"; the abstract says "1 to 3.7 seconds" across both models; Table 2 (secondary, BaseFold, seq 512) prints 1.65 s and Table 1 (primary, HyperKZG, seq 512) prints 3.22 s. The paper is internally inconsistent.
verification_time_note: Sec. 1.2 says ~4 s; abstract's range tops out at 3.7 s.
tokens_per_minute_note: NOT MEASURED. The distributed prover is SIMULATED, not built. Sec 5: "We simulate the performance of distributed DeepProve on 8 and 16 workers and a coordinator using our secondary machine configuration as the engineering effort to set up a distributed environment is non-trivial. We measure the maximum time to prove any chunk and add the time to transfer each individual proof back to the coordinator, assuming a LAN connection at 1 Gb/s." Table 6 is captioned "Simulated performance of distributed DeepProve", and the subsection closes: "We leave the full-fledged implementation of distributed DeepProve as future work." The abstract's "17 machines" = 16 workers + 1 coordinator. 10.64x over the 174.32 single-machine TPM at the same sequence length. Same sequence-length-not-decode-rate caveat as the rows above. DO NOT PLOT on the single-machine frontier. CORRECTED 2026-07-13: was tagged `reported`, which is what a measured number is called.
Notes
First system to prove full end-to-end LLM inference across all generated tokens of a prompt, rather than a single forward pass. Also reports a distributed prover reaching 1855 tokens/min. Bit width now confirmed at 12 (see quantization.bits): the 174 TPM is therefore at 12 bits vs zkGPT's 2.75 TPM at 16 bits, so the ~63x gap is NOT bit-width-normalized -- and the two also differ in context (512 vs 32) and in what is proven (full sequence certification vs a single token).
On the hardware
The headline 174/86 TPM figures are from the SECONDARY machine (faster clock, fewer cores). The primary machine peaks at 127 TPM (GPT-2) and 64 TPM (Gemma 3). Gemma 3 at seq 512 with HyperKZG OOMs on the secondary machine.
Quantization, in full
| bits | 12 |
|---|---|
| bits_note | "We run inference on the GPT-2 and Gemma 3 models using DeepProve using 12-bit quantization level" (Sec. 5, Experiments). Accuracy is also reported at 8 and 10 bits. Going 8 -> 12 bits costs only 0.8% extra prover time, because the number of lookup tables barely changes -- so the 12-bit accuracy is bought almost for free. |
| scheme | Symmetric affine quantization centered at zero (zero-point vanishes), for both weights and activations. Post-training static quantization with a calibration dataset; s = a / (2^(b-1) - 1) with a = max(|x_max|, |x_min|); activations clamped to [-a, a]. Mixed precision: residual layers use a larger bit length. Outlier smoothing via orthonormal rotation X*W = (X*M)(M^T*W), with M absorbed into the previous layer's weights. Scale factors are NOT restricted to powers of two -- a restriction DeepProve levels at zkTorch specifically, NOT at zkGPT. See conflict_flag. |
| rescale_mechanism | PRIMARY, Sec 4.3.2. y = floor( (Quant(2^f * eps) * x + 2^(f+delta-1)) / 2^(f+delta) ) - 2^(q-1). The arbitrary float scale eps is fixed-point-encoded as the integer Quant(2^f * eps), so the MULTIPLIER is arbitrary but the DIVISOR is a power of two -- i.e. a right shift, which the paper notes is "easier to prove". The shift is proven with a lookup argument over decomposable tables (Lasso/Shout-style): decompose x' into k-bit chunks, range-check each chunk, keep the q/k most significant. The +2^(f+delta-1) term is round-to-nearest (add a half before flooring). DeepProve's real novelty here is Case 2: it also proves requantization when the input is OUT of the expected range, via tables that emit -1/0/+1 to flag and correct overflowing elements. It states that the in-range case is "the only scenario considered by prior works [SLZ24 = zkLLM, QSL+25 = zkGPT]" -- i.e. those two ASSUME in-range inputs, which is accurate. CORRECTION, do not overclaim: DeepProve is the only PAPER with an explicit out-of-range protocol, NOT the only SYSTEM with out-of-range handling. Jolt Atlas does it too -- a saturating clamp (SatClamp: i64 accumulator clamped to i32) discharged by a prefix-suffix Shout sumcheck and FUSED INTO THE SAME REBASE SEAM as the rescale -- but only its SOURCE CODE says so; the paper never mentions it. The mechanisms differ: DeepProve DETECTS out-of-range with a proven predicate (zero-detector + sign-extractor tables blended to +/-2^(q-1)); Jolt SATURATES unconditionally with one clamp lookup. Proven predicate vs proven function. Jolt's coverage is partial where it matters: in ZK mode Add/Sub/Sum are proved UNCLAMPED, the ONNX Clamp op is an unproven passthrough (// TODO: Clamp), and the tanh/erf/sigmoid clamps are prover-side assert!s over unconstrained advice. |
| conflict_flag | UNRESOLVED -- and MIND THE ATTRIBUTION. DeepProve makes TWO DIFFERENT criticisms, one per system. They must not be merged, and this field previously merged them. (a) zkTorch gets the powers-of-two charge. DeepProve Sec 6, p.14: "the use of scaling factors is limited to powers-of-2, which incurs greater precision loss than the flexible quantization approach adopted in DeepProve." But zkTorch's OWN paper never says this -- "powers-of-two" does not occur anywhere in it. zkTorch states only that "any layer that multiplies two numbers must have a rescaling at some point after that layer to correct for the scale factor" (Sec 5) and that it uses "the similar scaling techniques proposed in ZKML" (Sec 6). So DeepProve's characterisation of zkTorch is inherited from ZKML or read off zkTorch's code, and is UNVERIFIABLE from zkTorch's text. Recorded, not resolved. (b) zkGPT gets a DIFFERENT charge -- not powers of two. DeepProve Sec 6, p.14: "Similar to zkTorch, zkGPT uses fixed-point multiplication with scaling factors for quantization ... using a fixed scaling factor without clamping can lead to situations where the values generated during inference are too large to be looked up, thus forcing a larger lookup table to commit." The charge is FIXED SCALE + NO CLAMPING (lookup-table blowup). The "similar to zkTorch" clause asserts similarity in USING fixed-point scale factors, not in restricting them to powers of two. zkGPT's own text is consistent with an arbitrary scale: Sec 2.1/4.1 computes qz = round((C1/C2) * qx * qy) with C1, C2 Q-bit integers "obtained by searching", approximating the float ratio Sx*Sy/Sz -- an arbitrary rational. The systems differ in HOW (DeepProve: integer multiplier + power-of-two shift; zkGPT: rational C1/C2 enforced by a double inequality, no division at all). CORRECTED 2026-07-13: this field previously said DeepProve accuses "zkGPT/zkTorch" jointly of powers-of-two. It does not. That merge propagated into content/numerics/index.md and content/zk-inference/quantization.md. |
| accuracy_retention | metric:
- cosine_similarity
- perplexity
dataset: Wikitext-2
gpt2:
fp32_perplexity: 49.2198
by_bits:
8:
cosine: 0.9967
perplexity: 60.9273
10:
cosine: 0.9993
perplexity: 50.0806
12:
cosine: 0.9966
perplexity: 49.4863
gemma3:
fp32_perplexity: 515.46
by_bits:
8:
cosine: 0.2035
perplexity: null
10:
cosine: 0.6998
perplexity: 1150.43
12:
cosine: 0.9587
perplexity: 454.72
summary: GPT-2 at 12 bits is within 0.54% of the fp32 perplexity baseline. |
| note | Gemma 3 collapses below 12 bits (cosine 0.20 at 8 bits). The paper attributes this to extremely large outlier activations plus extra RMSNorm layers in the transformer blocks, which block the orthonormal-rotation smoothing that works for GPT-2. Two oddities as printed: GPT-2's cosine similarity is slightly WORSE at 12 bits (0.9966) than at 10 (0.9993), and Gemma 3's 12-bit perplexity (454.72) is BETTER than its own fp32 baseline (515.46). Both reproduced verbatim from Table 4; neither is explained. |
Our reading
What is new
The contribution that matters is not cryptographic. It is a certification argument, and it is almost embarrassingly simple:
It just feeds the entire sequence of input and output tokens into the model at once, essentially performing a single inference step.
DeepProve · §3.1, Certification of the LLM Inference
Because a causal attention mask means the model's prediction at position t cannot see position
t+1, running one forward pass over the concatenated prompt-plus-answer sequence and checking
that each position predicts its successor certifies the whole generation. Proving cost drops from
quadratic in the token count to linear, really, to the cost of a single forward pass, whatever
the answer length. Everything else in the paper (per-layer PIOPs for requantization, RMSNorm,
grouped-query attention, RoPE; delayed requantization; outlier clamping) is competent
engineering in service of that one observation.
Two things genuinely distinguish it from the rest of the field. First, it is the only system here that proves round-by-round knowledge soundness for each component, "We formally prove that each component is round-by-round (knowledge) sound", which is precisely the property that makes the Fiat–Shamir compilation safe, and therefore the only direct answer anyone in this repo has given to the Fiat–Shamir/GKR attack. Second, it is a complete, open-source system rather than a proof-of-concept for one layer, which is a fair criticism of zkGPT and zkLLM and DeepProve is right to make it.
What it actually proves
A single forward pass over the concatenated prompt-and-answer sequence, under greedy (argmax) decoding, at 12-bit quantization, against a model the prover committed to.
Unpack that, because the abstract's "full LLM inference (i.e., for all generated tokens of a prompt)" invites a stronger reading than the system supports:
- The prover does not re-run the generation loop. It proves one pass and infers the rest from the attention mask. This is sound, and it is the paper's contribution, but the cost is one forward pass, exactly like zkGPT and zkLLM. What differs is what the paper counts against that cost (see below).
- Sampling is not proven. The certification depends on argmax giving determinism. The paper sketches a de-randomization via a public randomness beacon and then states plainly: "we do not include this in our implementation, same as all prior works on verifiable LLM inference." Every benchmarked number is greedy decoding. Real serving uses temperature and top-p. Note that CipherGPT, on the privacy side, does build a secure top-K sampling protocol, the zkML column still has no answer here.
- Zero-knowledge is not achieved. The paper defines zero-knowledge in an appendix and never claims its construction satisfies it; there is no hiding commitment, no masking polynomial, no ZK theorem. DeepProve is a succinct argument, not a zero-knowledge one. The weights are committed, not hidden. That is fine for the "lazy cloud swapped the model" threat, and fatal for the "model weights are a trade secret" motivation the introduction leans on.
- The quantization is assumed, not proven. The prover picks the calibration set, the scale factors, and the outlier-smoothing rotation. Nothing in the proof binds the committed integer model to any floating-point model a user would recognise.
What to distrust
The throughput metric is not the metric the baselines are measured with. DeepProve counts every token in the certified sequence against the cost of one forward-pass proof. Its comparison table converts zkGPT and ZKTorch to the same units by counting one token per proof, which is what those papers claim, but not what their provers cost. zkGPT's forward pass over its context would, under DeepProve's own certification argument, certify nearly all of those tokens too; the argument is a property of the GPT-2 architecture, not of DeepProve's prover. So the headline multiple is mostly the certification insight, and it is a free insight that any of these provers could adopt. The genuine prover-speed advantage is unmeasured, and is certainly far smaller.
The comparison table normalises the wrong variables. It lists CPU, thread count and RAM, and omits the two variables that actually decide the answer: sequence length (DeepProve's context is an order of magnitude longer than zkGPT's, and attention is quadratic in it) and bit width (12 vs 16). Both cut in DeepProve's favour once you see them.
The fastest prior system is excluded. "We exclude the comparison with zkLLM [SLZ24] since their performance numbers rely on GPU acceleration and are not replicable in CPU environments." The hardware objection is legitimate. But by zkGPT's own measurements, zkLLM's prover beats zkGPT's on GPT-2, so the abstract's speedup over "the state of the art" is measured against a system that, on the evidence of the very paper it cites, is not the state of the art.
The Gemma 3 accuracy table is not a usable accuracy claim, and the reason is not the one the paper gives. Its floating-point Gemma 3 perplexity baseline is an order of magnitude worse than its floating-point GPT-2 baseline on the same dataset under the same protocol, for a model with more than twice the parameters. Either the 256-token non-overlapping evaluation window suits Gemma 3 badly, or the baseline is misconfigured. You cannot measure quantization damage against a broken baseline, which is why the quantized model beats it.
Three, and the repo owner should know all of them:
- Peak throughput contradicts itself. §5 states the peak is 174 TPM "for GPT-2 at sequence lengths 512 using BaseFold", then in the next paragraph: "Remarkably, the best TPM we achieved in our experiments is 175, obtained for GPT-2 at sequence length 256 using the secondary machine." The same section also asserts throughput increases with sequence length. Table 3 uses 175.08. The abstract uses 174. Three numbers, two sequence lengths, one claimed trend that contradicts the claimed peak.
- Verification time. The abstract says 1–3.7 s; §1.2 says ~1.2 s (GPT-2) and ~4 s (Gemma 3); Table 1 prints 3.22 s and Table 2 prints 1.65 s for GPT-2 at the same sequence length on different machines. There is no single verification figure in this paper.
- Table 4's quantization curve is non-monotone in two places, GPT-2's cosine similarity is worse at 12 bits (0.9966) than at 10 (0.9993), and Gemma 3's 12-bit perplexity (454.72) is better than its own fp32 baseline (515.46). Neither is explained.
The memory criticism is self-inflicted. §1 dismisses ZKTorch because "the experimental evaluation of [CTK25] reports results for a machine with 4 TB RAM." §5 then concedes that DeepProve at its largest evaluated instance needs most of a workstation's memory, and that "proving models with more than 3 billion parameters, such as GPT-J, would likely require over a terabyte of RAM." That is the same objection, aimed at the same model class, arriving one paragraph too late.
Credit where it is due. The paper is unusually honest about its own overhead, it reports the plaintext CPU throughput of GPT-2 alongside its own, and concedes that for short-output queries the gap is close to three orders of magnitude. Most papers in this collection do not print the number that makes them look worst. This one does.
Citation neighbourhood
Builds on
Cited by, in this corpus
Edges are a proxy: paper A's text mentions B anywhere (body or bibliography). See the full graph.
Discussed in
Other recorded fields
claim_kind: seq