Papers · Numerics · SIRNN

SIRNN

SiRnn: A Math Library for Secure RNN Inference

Deevashwer Rathee, Mayank Rathee, Rahul Kranti Kiran Goli, Divya Gupta, Rahul Sharma, Nishanth Chandran, Aseem Rastogi

Venue
IEEE S&P 2021
Date
2021-05-10
Affiliation
Microsoft Research
Numbers from
primary
Open source
yes
Paper
https://eprint.iacr.org/2021/459
PDF
https://arxiv.org/pdf/2105.04236
Code
https://github.com/mpc-msri/EzPC
Quantization
not stated

Reported benchmarks

modelproving time scommunication mbnote
Google-30 (keyword spotting RNN)49.6450Against a uniform-64-bit ABY baseline: 3337 s and 259 GB. 67x faster, 574x less communication. The 423x in the abstract is the per-instance communication ratio for reciprocal square root against MP-SPDZ (6 KB vs 2535 KB).
Heads (SCUT Head, CNN+RNN)409.785500~3 million sigmoid/tanh calls and 2700 reciprocal-square-roots. The ABY baseline is intractable ("hundreds of terabytes"). First secure implementation of this model; no prior 2PC work supported rsqrt at all.

Approach

2PC, semi-honest -- LUT + Goldschmidt iteration for math functions, mixed bit-widths, digit decomposition

Notes

ONE OF THE THREE NODES THIS SOK'S HEADLINE FINDING RESTS ON, AND WE HAD NEVER READ IT. The bridge page has been arguing for months that "the only three papers cited by both columns are SIRNN, Cheetah and SecFloat, and all three are MPC numerics." We held SecFloat. SIRNN and Cheetah were names in a graph. Both are now read. What it is: specialized 2PC protocols for the math functions a network actually needs -- exponential, sigmoid, tanh, reciprocal, and reciprocal square root -- built on a single paradigm. Use a LOOKUP TABLE for a good initial approximation, then refine with GOLDSCHMIDT'S ITERATIONS. Support that with DIGIT DECOMPOSITION (split an l-bit value into l/d digits so you can index several small tables instead of one huge one, ~5x cheaper than garbled circuits) and MIXED BIT-WIDTHS (do not force every variable to 64 bits just because one operation needs it). Read that paragraph again and then read the zkML papers. Digit decomposition into small lookup tables, refined by an iterative approximation, with bit-widths chosen per operator. That is hao-et-al's exponential protocol, its reciprocal-square-root protocol, and its hyperparameters -- hao-et-al cites SIRNN five times in its BODY and inherits its iteration counts and lookup bit-lengths outright. It is also, in different vocabulary, what DeepProve and zkGPT rediscovered from first principles four years later. AND ITS MOTIVATION IS ZKPYTORCH'S CAUSAL CHAIN, WRITTEN IN 2021 IN MPC VOCABULARY: "the requirement of a high bitwidth even in one operation, coupled with the requirement of uniform bitwidths, raises the bitwidths of all variables and operations throughout an inference task, resulting in a communication blowup." Replace "communication blowup" with "field size" and you have the argument the numerics page opens with, four years early. THE FACT THAT SETTLES AN ARGUMENT ON THE NUMERICS PAGE: SIRNN'S TRUNCATION IS EXACT. All four of its truncation flavours -- logical shift, arithmetic shift, truncate-and-reduce, and C-style division by a power of two -- compute the wrap/borrow terms exactly. There is no "fails with small probability" anywhere in the paper, and truncate-and-reduce costs ~lambda(s+1) bits, ROUGHLY 4.5x LESS than a garbled circuit. So the MPC world did not only have the cheap-but-probabilistic truncation that ZK cannot borrow; it also had the exact one, and the exact one is the one that is cheap. Our claim that "probabilistic truncation is the exception that has been mistaken for the rule" now has its primary source.

Quantization, in full

bits
None
bits_note
The whole point is that there is no single bit width -- mixed, non-uniform bit-widths are the contribution. Formally verified against GNU MPFR at bit-width 16 by exhaustive testing.
scheme
Fixed-point, exact truncation (four flavours). Math functions via LUT initial approximation + Goldschmidt iterations. Precision is reported as ULP error, not as an accuracy delta: exponential 3 ULP, sigmoid 3, tanh 4, reciprocal square root 4. "Intel's SVML also provides math implementations with 4 ULP error."
accuracy_retention
"SIRNN has no loss in model accuracy." Its foil: SecureML/ABY2.0's 3-piece linear spline sigmoid carries 1547 ULPs of error and drops Google-30 keyword-spotting accuracy from 84.4% to 60.95%. Cheap approximations of the non-linearities are how you destroy a model, and this is the cleanest measurement of it anywhere in the corpus.

Our reading

For as long as this SoK has existed, its headline finding has rested on three papers: the only three nodes cited by both columns are SIRNN, Cheetah and SecFloat, and all three are MPC numerics.

We held SecFloat. SIRNN and Cheetah were names in a graph. We had never read either. This page is the overdue half of that.

What it is

Specialized 2PC protocols for the math functions a network actually needs, exponential, sigmoid, tanh, reciprocal, and reciprocal square root, all built on one paradigm:

Use a lookup table for a good initial approximation, then refine with Goldschmidt's iterations.

Supporting that are two ideas that carry the whole paper. Digit decomposition splits an $\ell$-bit value into $\ell/d$ digits so you can index several small tables instead of one enormous one, about 5× cheaper than a garbled circuit. And mixed bit-widths: do not force every variable to 64 bits just because one operation needs it.

Now read that paragraph again, and then read the zkML papers.

Digit decomposition into small lookup tables, refined by an iterative approximation, with bit-widths chosen per operator. That is Hao et al.'s exponential protocol and its reciprocal-square-root protocol, it cites SIRNN five times in its body and inherits its iteration counts and lookup bit-lengths outright. It is also, in different vocabulary, what DeepProve and zkGPT rediscovered from first principles four years later, and what Bionetta derived from scratch in an appendix in December 2025.

The sentence that should have been in our numerics page three years ago

the requirement of a high bitwidth even in one operation, coupled with the requirement of uniform bitwidths, raises the bitwidths of all variables and operations throughout an inference task, resulting in a communication blowup

SIRNN · §II, Motivation

Replace communication blowup with field size and you have zkPyTorch's causal chain, the one the numerics page opens with, stated in 2021, in MPC vocabulary, by people who then went and solved it.

The fact that settles an argument

The numerics page argues that ZK cannot borrow MPC's cheap probabilistic truncation, because a failure that is rare on random data is not rare at all when the prover chooses the input, and it closes by insisting that probabilistic truncation is the exception that has been mistaken for the rule.

SIRNN is the primary source for that, and we were making the argument without it.

SIRNN's truncation is exact. All four flavours, logical shift, arithmetic shift, truncate-and-reduce, and C-style division by a power of two, compute the wrap and borrow terms exactly. There is no "fails with small probability" anywhere in the paper. And the exact one is cheap: truncate-and-reduce costs about $\lambda(s+1)$ bits, roughly 4.5× less than a garbled circuit.

So the MPC world did not merely have the cheap-but-probabilistic truncation that ZK must refuse. It also had the exact one, and the exact one is the one that is fast. That is precisely the technique the ZK papers borrowed, and precisely the reason the borrowing was possible at all.

What to distrust

Very little, which is unusual for this collection. The precision claims are not asserted, they are formally verified: exhaustively tested at bit-width 16 against GNU MPFR, reporting worst-case ULP error (exponential 3, sigmoid 3, tanh 4, reciprocal square root 4). The paper's own benchmark is that "Intel's SVML also provides math implementations with 4 ULP error."

The one thing to carry forward is its foil, because it is the sharpest measurement in the corpus of what cheap non-linearities cost a model. SecureML and ABY2.0's three-piece linear spline sigmoid carries 1547 ULPs of error, and drops Google-30 keyword-spotting accuracy from 84.4% to 60.95%.

Approximating the activation is how you destroy the network. Both columns keep learning this separately.

Citation neighbourhood

Edges are a proxy: paper A's text mentions B anywhere (body or bibliography). See the full graph.