3PC secret sharing (2-out-of-3 replicated), semi-honest, honest majority

THE PAPER THAT SHOWS THE BERT CEILING IS A 2PC CEILING. Iron needs 280.99 GB and 216 minutes for one BERT-base inference; PUMA does LLaMA-7B, sixty times the parameters, in 200 seconds and 1.79 GB. Nothing about privacy forbids scale. What forbids scale is dishonest-majority two-party computation. Technique: piecewise low-degree polynomial approximations fitted with numpy.polyfit for GeLU (4 pieces, split at -4/-1.95/3) and SiLU; a clipped Taylor "negExp" for softmax that shifts every exponent input negative and replaces n divisions with one reciprocal and n multiplies; a secure Embedding protocol that builds the one-hot vector inside MPC (so unmodified HuggingFace checkpoints load); and a LayerNorm that computes sigma^-1/2 once and broadcasts. Accuracy is genuinely preserved on the models where it is measured -- GLUE accuracy within 0.011 of plaintext, GPT-2 perplexity within 0.02. WHAT TO DISTRUST, and there is a lot. (1) THE TITLE'S NUMBER IS IN NO TABLE. Abstract says "around 5 minutes to generate 1 token". Table 6 says 200.473 s (3.3 min) for 8 input tokens and 1 output token, and 364.527 s (6.1 min) for 2 output tokens. Three framings of one measurement, and the one on the cover is not the one in the table. (2) The LLaMA-7B prompts are FOUR AND EIGHT TOKENS. No realistic prompt length is ever run on LLaMA-7B, and softmax cost is superlinear in sequence length. (3) No numeric accuracy or perplexity is reported for LLaMA-7B at all. The evidence is one prompt in an appendix, "with fixed randomness". (4) No KV cache is described anywhere. Going from 1 to 2 output tokens costs 1.82x, which is close to a full re-run of the forward pass. (5) Efficiency gains SHRINK with sequence length: at GPT-2 with 256 inputs, PUMA's communication is 0.936x MPCFormer's, i.e. WORSE than the baseline it beats everywhere else. (6) No WAN numbers. The LLaMA row assumes a 20 GB/s datacenter link.

3PC, SEMI-HONEST, HONEST MAJORITY -- at most one of three non-colluding computing parties is corrupted. Model owner and client secret-share the weights and the input to P0/P1/P2; the result is reconstructed to the client. This is a materially stronger trust assumption than the 2PC systems in this SoK (iron, ciphergpt, bolt, nimbus, cheetah), which tolerate a dishonest majority, and PUMA picks it explicitly because it "has the highest concrete efficiency". Do not put a 3PC latency next to a 2PC latency without saying so. Out of scope by the paper's own statement: "PUMA cannot defend against attacks based on inference results".

PUMA is the paper that shows the BERT ceiling is a 2PC ceiling, not a privacy ceiling.

Iron needs 280.99 GB and 216 minutes for one BERT-base inference. PUMA runs LLaMA-7B, sixty times the parameters, in 200 seconds and 1.79 GB. Nothing about privacy forbids scale. What forbids scale is dishonest-majority two-party computation.

The threat model, which is the whole story

PUMA is secure against a semi-honest adversary that corrupts no more than one of the three computing parties.

PUMA · §3.3, Threat Model

3PC, semi-honest, honest majority. Model owner and client secret-share the weights and the input to three non-colluding servers; the result is reconstructed to the client. PUMA picks this setting explicitly because it "has the highest concrete efficiency."

That third party is not free, and it is the reason a PUMA latency cannot be placed next to an Iron latency without a sentence of explanation. But it is also not exotic, it is the standard honest-majority MPC assumption, and an entire branch of the field runs on it. Our corpus simply contained none of that branch.

What is new

The interesting parts are all about the same three operators the verifiability column fights.

GeLU gets a four-piece low-degree polynomial (split at $-4$, $-1.95$, $3$), fitted with numpy.polyfit and evaluated with three secure comparisons, rather than composing tanh and sigmoid out of generic exponential and reciprocal protocols.

Softmax gets a clipped Taylor "negExp": add $\varepsilon$ so every exponent input is strictly negative, clip below $-14$, evaluate $(1 + x/2^t)^{2^t}$ by repeated squaring, and replace $n$ divisions with one reciprocal and $n$ multiplies. The max-subtraction, the range clipping, the division-avoidance, every move here has a twin in zkGPT or DeepProve, reached independently.

Secure Embedding builds the one-hot vector inside MPC via secure equality tests, which is why unmodified HuggingFace checkpoints load. MPCFormer had pushed one-hot generation to the client.

And the accuracy holds where it is measured: GLUE within 0.011 of plaintext, GPT-2 perplexity within 0.02.

What to distrust, and there is a lot

The LLaMA-7B prompts are four and eight tokens. No realistic prompt length is ever run on LLaMA-7B, and softmax cost is superlinear in sequence length, so the number does not extrapolate.

The LLaMA-7B row is on completely different hardware from everything else in the paper. BERT and GPT-2 run on 32-vCPU / 128 GB machines over a 5 Gbps link. LLaMA-7B runs on 128-thread, 1 TB RAM machines over a 20 GB/s link. The headline assumes a datacenter.

No accuracy number for LLaMA-7B exists. The GLUE and perplexity tables cover BERT and GPT-2. The evidence for the model in the title is one prompt in an appendix, "with fixed randomness."

No KV cache. The paper never mentions one. Going from one output token to two costs 1.82×, close to a full re-run of the forward pass. Whatever this is, it is not efficient decoding, and it is the same wall CipherGPT hit.

And the win evaporates at length. At GPT-2 with 256 input tokens, PUMA's communication is 0.936× MPCFormer's, that is, worse than the baseline it beats everywhere else. The paper reports this. Nobody who cites the five-minute figure does.