Clamping
saturating out-of-range activations
The algorithm
Outliers exceed the calibrated range. Rather than let them corrupt the scale — or fall outside the lookup table's domain and become unprovable — saturate them at the boundary.
Why it resists a proof
The obvious protocol, one table mapping every possible input to its clamped output, has a domain the size of the pre-shift value range. You must express the clamp algebraically instead.
Cryptographic toolbox
- Zero-indicator polynomials evaluated by sumcheck
- Sign-indicator polynomials for signed ranges
- Fusing the clamp into the requantization decomposition
How each scheme proves it
Never build a clamp table — and the code confirms it literally: there is no Clamp table type. Clamping lives inside the Requantise/ReLU tables (their apply is input.clamp) plus a zero-chunk selector folded into the sumcheck.
The simplest solution would be to have another lookup table that takes as input Out(x̄) and outputs Clamp(x̄). In practice though this clamping lookup table is too large and so DeepProve uses two different approaches based on whether the value is expected to be signed or not.
DeepProve · Appendix C
Since re-quantization and non-linear functions are usually proven via lookup arguments, using a fixed scaling factor without clamping can lead to situations where the values generated during inference are too large to be looked up, thus forcing a larger lookup table to commit.
DeepProve · §6
- Unsigned. Decompose into
ZeroInchunks, each paired with a multilinearZeroOutthat is 1 iff its chunk is zero. Then sumcheck $\mathrm{Out}(r)=\sum_b \mathrm{eq}(r,b)\big[(\prod_j \mathrm{ZeroOut}_j)\mathrm{Value}(b) + (1-\prod_j \mathrm{ZeroOut}_j)\mathrm{CMax}\big]$. The product is 1 exactly when no high chunk fired. - Signed. Add a
SgnIn/SgnOutpair returning $-1/0/{+}1$, so the clamp target interpolates: $\frac{\mathrm{CMax}-\mathrm{CMin}}{2}\mathrm{SgnOut} + \frac{\mathrm{CMax}+\mathrm{CMin}}{2}$. - Because clamping is folded into the requantization lookup, the downstream non-linearity receives the unclamped linear output — strictly more accurate than clamping first.
- This is exactly the gap DeepProve identifies in zkGPT: a fixed scaling factor without clamping means inference can produce values too large to look up, forcing a larger committed table.
- In code the eight
TableTypes areExp, GeLU, ReLU, SiLU, Requantise, ShiftCheck, SignedZeroCheck, ZeroCheck— none of them is a clamp table. The Requantise and ReLU tables clamp inside their ownapply_op(input.clamp(min,max)); the out-of-range selector is the product ofZeroCheckchunk outputs. See Lookup arguments & range checks.
Two clamps, and only one is proven. The fused saturating clamp after a rebase is a real SatClamp<64> lookup; the standalone ONNX Clamp operator is an unproven passthrough.
- The fused clamp is real. After a rebase,
output = SatClamp(rescaled)is discharged by a 64-bitSatClampTablelookup via prefix-suffix Shout, with the pre-clamp value carried asClampAccadvice and one-hot read-address checks. Shared by Einsum / Mul / Square / Cube / Add / Sub / Sum. - The table is never materialized at $2^{64}$: its MLE is a prefix-suffix combination of sign-extension-detection prefixes and saturating-value suffixes.
- But the standalone
Clampop is a TODO dummy that forwards its operand claim with no clamp check at all — a distinct code path from the fusedSatClamp.
// TODO: Clamp // Currently this is just a dummy implementation that passes down an operand claim for the rest of the proof system let accessor = AccOpeningAccessor::new(&mut prover.accumulator, node); let (opening_point, _claim) = accessor.get_reduced_opening();
A bare Clamp ONNX node is unproven (ops/clamp.rs passthrough). Only the fused saturating clamp is sound. Also: SatClampTable's prefix-suffix combine hard-codes the i32 split and asserts XLEN==64 only via debug_assert — a release build wouldn't catch a wrong-XLEN misuse.