Sigma
Sigma: Secure GPT Inference with Function Secret Sharing
Reported benchmarks
| model | params | context window | proving time s | communication mb | accuracy | note |
|---|---|---|---|---|---|---|
| Llama2-13B | 13B | 128 | 37.59 | 18900 | 72.99% Lambada (vs 73.14% fp32) | Online phase only. Preprocessing: 419.01 GB of FSS keys, 40.57 s to generate, 356.61 s to transfer. True wall clock ~435 s. This is the largest model in the privacy column and it is the same size as the largest model in the verifiability column (zkLLM's LLaMA-2-13B). |
| Llama2-7B | 7B | 128 | 23.09 | 12070 | 69.92% Lambada (vs 70.17% fp32) | Online only. 255.41 GB of keys; 217.37 s to transfer them. CrypTen cannot run this model. |
| GPT-Neo | 1.3B | 128 | 7.19 | 4030 | — | — |
| GPT-2 | 124M | 128 | 1.51 | 820 | — | — |
| BERT-base | 110M | 128 | 1.72 | 990 | — | Compare the anchor this SoK has been quoting for three years: Iron, BERT-base, 280.99 GB and 216 minutes. Sigma: 0.99 GB and 1.72 seconds online (16.84 GB of keys, 14.33 s to transfer). Same model, same threat-model family, four years apart. |
| BERT-large | 330M | 128 | 4.44 | 2640 | — | — |
Approach
Function secret sharing (FSS) in the preprocessing model, GPU-accelerated
Notes
THE PAPER THAT BREAKS THE "PRIVACY CAPS AT BERT" CLAIM FROM INSIDE 2PC. Sigma runs Llama2-13B -- the same scale zkLLM proves in the verifiability column -- with a 37.59 s online phase and 18.90 GB of online communication. Iron, which this SoK uses as the anchor for the whole privacy column, needs 280.99 GB and 216 minutes for one BERT-base. Sigma does BERT-base in 1.72 s and 0.99 GB. The mechanism is a different branch of the field that our corpus contained NONE of: function secret sharing, moved onto a GPU. Its two ideas are worth having. First, faithful (non-local) truncation protocols in FSS -- it explicitly rejects CrypTen's local truncations as known- insecure. Second, LUT-shrinking by domain knowledge: rather than Pika's 2^50 table, it observes that delta(x) = ReLU(x) - GeLU(x) is zero outside (-4, 4), so a 256-ENTRY table suffices for GeLU over 50-bit values. SiLU gets 1024 entries. That is the same "exploit the operator's actual range" move the zkML lookup papers make, arrived at independently. WHAT TO DISTRUST, AND IT IS THE HEADLINE. THE ADVERTISED NUMBERS ARE ONLINE-ONLY AND THE PREPROCESSING IS ENORMOUS. Llama2-13B needs 419.01 GB OF FSS KEYS, 40.57 s to generate them, and 356.61 s just to TRANSFER them over the same 9.4 Gbps LAN -- against the 37.59 s online figure on the cover. Real wall-clock is roughly 435 s, about 11x the advertised number. Llama2-7B: 255.41 GB of keys, 217.37 s of transfer, 23.09 s online. The keys are one-shot and shape-specific; there is no amortization story for a second query. Also: CrypTen OOMs on 7B and 13B, so the models the title is about have NO BASELINE AT ALL -- the 12-19x speedup range is measured on the models nobody cares about. And Table 4 reports BERT-base/MRPC at 87.25% secure vs 84.31% plaintext, i.e. the encrypted version scores THREE POINTS BETTER than the model it is emulating, on a 408-example validation set. That is noise, and it is in the table that is supposed to demonstrate faithfulness. Single forward pass only. The strings "KV cache", "autoregressive" and "decoding" do not appear. Generating N tokens means N full runs, each with a fresh key set -- so at 419 GB of keys per run, generation is not a thing this system does.
Threat model
2PC, SEMI-HONEST, static, IN THE PREPROCESSING MODEL WITH A TRUSTED DEALER -- and that last clause is the one that matters. The online phase is genuinely two-party and dishonest-majority; but the correlated randomness (the FSS keys) is generated by a trusted third party, which Sigma chooses out of three options it lists. So it is not a peer of iron/bolt/cheetah, which need no dealer, and it is not a peer of puma/mosformer, whose third party is untrusted-but-one- of-three. It is its own trust model and this SoK previously had no slot for it. Malicious security is explicitly future work.
Quantization, in full
| bits | None |
|---|---|
| bits_note | Not a single bit width: Sigma runs each model at a bespoke ring size chosen per model (37 bits for BERT-tiny, 50 for BERT-base/large and GPT2, 51 for GPT-Neo, 48 for Llama2), with fixed-point precision f = 12. The "effective bitwidth" analysis is one of its contributions. |
| accuracy_retention | Measured as task accuracy vs 32-bit float PyTorch. Llama2-7B Lambada 69.92% vs 70.17%; Llama2-13B 72.99% vs 73.14%; GPT-Neo 57.81% vs 57.46%; BERT-base QNLI 91.63% vs 91.60%. No perplexity is reported anywhere. BERT-base/MRPC comes out 2.94 points ABOVE plaintext, which is a 408-sample artifact and is reported without comment. |
Our reading
Sigma is the paper that ends the claim this SoK has been making for its entire existence: that the privacy column caps out at BERT.
It does not. Sigma runs Llama2-13B, the same model zkLLM proves in the verifiability column, with a 37.6-second online phase and 18.9 GB of online communication. Iron, the anchor this repo has quoted for BERT-base, needs 280.99 GB and 216 minutes. Sigma does BERT-base in 1.72 seconds and 0.99 GB.
Four years, same threat-model family, four orders of magnitude.
What is new
Function secret sharing, on a GPU. Our corpus contained no FSS papers at all, and that omission is most of why the ceiling looked real. FSS moves the work into a preprocessing phase that produces correlated randomness (the "keys"), after which the online phase is startlingly cheap.
Two ideas inside it are worth having even if you never touch FSS.
Faithful truncation. Sigma builds non-local truncation protocols in FSS and explicitly rejects CrypTen's local truncations, which it cites as known-insecure. This is the same fault line the numerics page draws between exact and probabilistic rescaling, and Sigma lands on the exact side.
Lookup-table shrinking by domain knowledge, which zkML would recognize instantly. Rather than Pika's $2^{50}$-entry table, Sigma observes that $\delta(x) = \mathrm{ReLU}(x) - \mathrm{GeLU}(x)$ is zero outside $(-4, 4)$, so a 256-entry table suffices for GeLU over 50-bit values. SiLU gets 1024 entries. That is exactly Jolt Atlas's neural teleportation and DeepProve's decomposable tables: shrink the domain, shrink the table. Two literatures, one move, no citation between them.
What it actually proves
Nothing. This is the privacy column, there is no proof object, no verifier, no third party who can check anything. Sigma hides the client's input from the model owner and the model from the client, against a semi-honest adversary, and that is all.
Read threat_model in papers.yml before citing it against anyone, because the trust model is
genuinely novel in this corpus and does not fit either existing slot:
- The online phase is 2PC, dishonest-majority. Like Cheetah, Iron, BOLT.
- The preprocessing is done by a trusted dealer. Sigma lists three ways to generate the correlated randomness and picks the one with a trusted third party.
So it is neither a peer of Iron (which needs no dealer) nor of PUMA (whose third party is untrusted-but-one-of-three). It is its own point in the space, and the honest way to state the corrected finding is: pure dishonest-majority 2PC with no dealer caps out at BERT-class. Private inference does not.
What to distrust
Llama2-13B needs 419.01 GB of FSS keys. Generating them takes 40.57 s on a GPU dealer, and transferring them takes 356.61 seconds over the same 9.4 Gbps LAN the online phase runs on.
Against a headline of 37.59 s.
True wall-clock is roughly 435 seconds, about 11× the advertised figure. Llama2-7B: 255 GB of keys, 217 s of transfer, against 23 s online. The paper is not hiding this; Table 9 is in the appendix and it is honest. But every number on the cover is the online column, and the online column is not what a user waits for.
And the keys are one-shot and shape-specific. There is no amortization across queries. A second inference means a second 419 GB.
The models the title is about have no baseline. CrypTen OOMs on 7B and 13B. So the "12–19× speedup" band is measured on BERT and GPT-2, the models nobody needed FSS for, and the Llama rows, which are the entire contribution, are compared against nothing.
Table 4 says the encrypted model is better than the model. BERT-base on MRPC: PyTorch 84.31%, Sigma 87.25%. A faithful fixed-point emulation scoring three points above the float model it emulates is noise on a 408-example validation set, and it appears, without comment, in the table whose job is to demonstrate faithfulness. It does not undermine the paper, but it tells you how much weight that table can bear.
It is a single forward pass. The strings KV cache, autoregressive and decoding do not appear. Generating $N$ tokens means $N$ full runs, each with a fresh key set, so at 419 GB per run, generation is not something this system does. The decode problem is unsolved on this side of the 2×2 too.
Citation neighbourhood
Edges are a proxy: paper A's text mentions B anywhere (body or bibliography). See the full graph.