Bootstrapping is All You Need
Bootstrapping is All You Need: Secure Transformer Inference via Improved CKKS Functional Bootstrapping
Reported benchmarks
| models | inference latency s | communication mb | accuracy | network | vs sota |
|---|---|---|---|---|---|
| BERT-base (BERT-DyT -- LayerNorm replaced by Dynamic Tanh, distilled from BERT) | 349.5 | 16.1 | -1.73 / -0.57 / -0.98 points vs plaintext BERT on RTE / SST-2 / MRPC | n/a -- non-interactive FHE. The 16.1 MB is a one-shot ciphertext upload/download, not an interactive transcript. | 1.9x faster (349.5s vs 662.3s), 3x less communication (16.1MB vs 48.3MB) -- vs MOAI |
inference_latency_note: End-to-end FHE inference latency. Sec 6: "Our framework takes 349.5s to complete the entire inference, which is 1.9x faster than MOAI." Table 4 ("Runtime breakdown amortized over 256 inputs") sums to exactly 349.5 s across homomorphic evaluation steps only -- there is no commit, no prove and no verify anywhere in the paper. Seq len 128; 12 layers, 12 heads, hidden 768, FFN 3072; "each query is processed with a batch size of 256 sequences". Intel Xeon Platinum 8358 @ 2.60 GHz, 64 cores; OpenFHE v1.5.1 + OpenMP.
communication_note: per query, amortized over 256 inputs (Sec 6 / Fig 5)
accuracy_note: MEASURED, Table 3, and previously unrecorded. Plaintext BERT and BERT-DyT are identical (71.12 / 93.00 / 86.76); encrypted BERT-DyT gives 69.39 / 92.43 / 85.78. The paper's gloss: "our framework introduces virtually no additional accuracy loss." Both error sources (the DyT substitution AND the CKKS approximation) are folded into that column.
Approach
DIFFERENT sub-approach from the 2PC ones above: pure FHE over CKKS. Bootstrapping is the main NISTI bottleneck (66.8% of runtime in the SOTA). Their Functional Bootstrapping (FBS) FUSES as many operations as possible into each bootstrap step, cutting the prescribed multiplicative depth (which is what drives ciphertext size / cost). Three pieces: (1) a TRIGONOMETRIC minimax approximation for FBS (a trigonometric Remez algorithm; better worst-case L-infinity precision on non-linear layers than the Fourier/L2 SOTA); (2) FUNCTIONAL S2C -- fuse linear layers (y=xW and y=xW+b) into the slot-to-coefficient transform inside FBS, so linear layers cost nothing separately; (3) batch-S2C + interleaved packing / lazy key switching / hoisting.
Notes
It is FHE (non-interactive, single-party on ciphertext), not 2PC like the others. Its tiny communication -- 16.1 MB against BOLT's 25.74 GB (with word elimination) or 59.61 GB (without), BOLT Table 3 -- is roughly a 1,600x gap, and that is the whole point of FHE: it trades interaction and communication for heavy homomorphic compute. CAUTION on the shorthand "the 2PC systems' hundreds of GB", which this note used to assert: in this cell that is true of exactly ONE figure, Iron's 280.99 GB -- and that figure is BOLT's RE-IMPLEMENTATION of Iron under a WAN, not Iron's own self-report (Iron's paper prints no numeric end-to-end communication total at all; its Fig 4 is a log-scale bar chart with ratio annotations only). BOLT itself is at tens of GB, and CipherGPT reports ~15 GB per generated token. The true comparison is stronger than the false one, so there is no reason to keep the false one. Note the transformer it benchmarks uses DyT (Dynamic Tanh) in place of LayerNorm. Same goal as the 2PC systems (private inference), different cryptographic route.
Security model
FHE (CKKS), single-party, NON-interactive -- NOT 2PC. They call the setting NISTI.
Terminology
NISTI = Non-Interactive Secure Transformer Inference (client encrypts input under FHE, server evaluates on ciphertext, returns ciphertext).
Our reading
What is new
This is the one FHE paper in the privacy cluster, and the setting is genuinely different from the 2PC systems around it. The client encrypts under CKKS, the server evaluates the whole transformer on ciphertext, and returns a ciphertext. No interaction, no rounds, no client presence during inference. The authors call it NISTI, non-interactive secure transformer inference, and it is the only entry in either column of this repo where the client can go offline.
Bootstrapping dominates that setting, so the paper's thesis is: stop treating bootstrapping as overhead and start fusing useful work into it. Three pieces:
A trigonometric minimax approximation for functional bootstrapping. The prior state of the art approximates the target function with a Fourier series, which is optimal under L2 but does not minimize worst-case error, and worst-case error is what you care about when a single bad activation can flip a classification. The classical Remez algorithm minimizes L∞ but only over a polynomial basis, and functional bootstrapping needs a trigonometric one. The paper proves a trigonometric minimax approximation exists and derives a trigonometric Remez algorithm to compute it. This is the real contribution and it is a proper theorem.
Functional S2C. Bootstrapping already performs a slot-to-coefficient transform, which is a linear map. A transformer's linear layers are also linear maps. So fold the weight matrix into the S2C transform and the linear layers cost nothing separately. This is a lovely observation.
Batch-S2C, interleaved packing, lazy key switching, hoisting, engineering.
What it actually proves
Nothing, no proof, and no integrity guarantee of any kind. What it computes, privately and non-interactively, is BERT-DyT, not BERT.
For our framework, we replace LayerNorm with Dynamic Tanh (DyT). We distill BERT-DyT from the original BERT, treating the latter as the teacher model.
Bootstrapping is All You Need · §6, Evaluation
That substitution is not incidental. It is where most of the headline speedup comes from.
The other thing to be precise about: both headline numbers are amortized over a batch of 256 inputs. The runtime breakdown table says so in its caption; the communication figure says so in the abstract. No single-query (batch-1) latency or bandwidth is reported anywhere. The README currently sets this system's tiny communication figure against the 2PC systems' hundreds of gigabytes as "the whole FHE trade", but those 2PC figures are per inference, and this one is per inference amortized over 256. FHE still wins the communication axis, and by a lot, but not by the margin that comparison implies, and a user sending one prompt cannot amortize over anything.
What to distrust
The headline speedup is largely a model substitution the baseline does not get, and the paper's own table caption admits it.
Table 4's caption reads, in full: "Runtime breakdown amortized over 256 inputs. MOAI still uses LayerNorm instead of Dynamic Tanh."
So the comparison is between two different networks. Look at what that buys, in the paper's own numbers:
| Operation | Ours | MOAI (baseline) |
|---|---|---|
X = α · tanh(β · (E · W₂ + b₂)) + γ — the DyT layer |
76.4 s | 217.4 s |
E = GELU(D · W₁ + b₁) |
141.1 s | 43.4 s |
| Total | 349.5 s | 662.3 s |
The single row corresponding to the operation they replaced in the model accounts for ~141 s of the ~313 s total saving, roughly 45% of the entire speedup, from the one change the baseline was not allowed to make.
And in the same table, their GELU is over three times slower than the baseline's. That regression is printed and never discussed anywhere in the text.
The paper's three cryptographic contributions may well be sound and useful. But the experiment as constructed cannot tell you what they are worth, because it never runs the new bootstrapping machinery on a LayerNorm network, the one ablation that would separate the cryptography from the architecture surgery. This is BOLT's word-elimination problem in a different key: a real cryptographic contribution, and a headline number that quietly bundles a model change the baseline never receives.
"Virtually no additional accuracy loss" is not what the accuracy table shows. The text says the framework "introduces virtually no additional accuracy loss, achieving nearly identical accuracy to BERT-DyT across all three datasets." The encrypted evaluation loses roughly a point on two GLUE tasks and nearly two points on the third (RTE). That is the largest accuracy loss in the privacy cluster, Nimbus reports a fraction of a percent, Iron under a third of a percent, BOLT around one percent, and it is described as negligible.
The distilled model reports accuracy identical to its teacher, to two decimal places, on all three datasets. Across test sets of 277, 872 and 408 examples, replacing LayerNorm with Dynamic Tanh and distilling from scratch is reported as changing the number of correct predictions by exactly zero, three times. That is possible. It is also the kind of coincidence that deserves a sentence of explanation, and gets none.
Only three GLUE tasks, all classification, all short-sequence. Nimbus reports eight.
Where the paper is strong: the trigonometric minimax result is a genuine theorem with a real motivation (L∞ is the right norm for this application and the prior work used the wrong one), the functional-S2C insight is elegant, and the non-interactive setting is a materially different product from everything else in this repo. If someone re-ran this on a LayerNorm network and the speedup survived, it would be the most interesting paper in the privacy cluster. As published, that experiment does not exist.
Citation neighbourhood
Cited by, in this corpus
- none recorded
Edges are a proxy: paper A's text mentions B anywhere (body or bibliography). See the full graph.
Discussed in
Other recorded fields
pdf_available: true