Secure Multi-Party Computation Protocols for Private ML Inference
Math-based protocols let AI models process data without ever seeing it in plain form.
Secure multi-party computation solves a problem that sounds almost paradoxical: how do you let a model process data it never actually sees? A client holds sensitive input, a server holds a proprietary model, and neither wants to hand the other its data in the clear. SMPC splits the computation across multiple parties so no single one of them ever reconstructs the plaintext, whether that's the input, the weights, or anything sitting in between. This piece walks through how that works, where the field stands right now, and what still separates the research frontier from something you'd trust with real patient records or account data.
The urgency isn't abstract. Every time someone sends a prompt to a hosted large language model, that prompt sits in plaintext on someone else's servers, at least for a moment and often longer. In healthcare and financial services, that's the reason a whole category of AI applications never gets built: compliance teams won't let regulated data anywhere near third-party compute, no matter how good the model is. The inference market already runs into the billions of dollars, and the slice touching regulated data is exactly the slice that turns SMPC into a business question instead of an academic one. Access controls and contracts rely on trust and enforcement after the fact, whatever a vendor happens to promise in a terms-of-service document. The point of this research is to make privacy something math enforces directly.
How secret sharing, garbled circuits, and homomorphic encryption each handle a different part of a neural network
Strip a neural network down and you get an alternating pattern: linear operations like matrix multiplication, then non-linear operations like ReLU, then more linear operations, on and on. That structure lines up almost exactly with the strengths of the three main cryptographic tools in this space. Once you see the match, hybrid systems stop looking like a compromise, and they start looking like the obvious design, the one you'd land on if you sat down and actually mapped tool to task.
Additive secret sharing, specifically the 2-out-of-3 replicated version, splits a value across parties so no subset of them can put it back together alone. It's cheap and fast for linear math, which is why it carries the matrix multiplications that make up most of a neural network's raw compute. Garbled circuits handle arbitrary boolean functions in a fixed number of communication rounds, a good fit for non-linear operations like ReLU or max-pooling. The catch: garbled circuits carry a heavy communication cost per logic gate, so using them for linear layers too would burn bandwidth for nothing.
Homomorphic encryption, particularly CKKS for approximate arithmetic and BFV for integer work, lets you compute directly on encrypted data without decrypting mid-computation. That opens the door to non-interactive inference: the client sends an encrypted input and waits for an encrypted result, no back-and-forth needed. But HE schemes eat memory and compute time, and both get worse as circuit depth grows.
None of these three wins across the board, and I don't think that's a flaw in the field so much as a fact about the problem. Each got built for a specific job, and private inference has enough variety in it that you end up needing more than one tool.
A pattern shows up across almost every major system: preprocessing-based MPC. Push as much work as possible into an offline phase, before the real input even shows up, generating correlated randomness (commonly called Beaver triples) that the online phase can burn through fast once the input arrives. Function secret sharing (FSS) takes this further and applies it specifically to non-linear functions like GeLU and Softmax, moving their expensive parts into an offline key-generation step. That's become central to Transformer inference, where those functions show up constantly and would otherwise dominate the online cost.
Why hybrid protocols that combine HE and MPC have become the dominant architecture for practical inference
Given that division of labor, the natural design runs linear layers under HE and hands non-linear activations to MPC or garbled circuits, letting each primitive do the part it's built for. Simple enough on paper. In practice, the seams between these two worlds are where most of the cost hides, and that's where a lot of the actual engineering happens.
Switching a value from an HE ciphertext into an MPC secret share, or back again, costs real time and bandwidth. A protocol that introduces these conversions carelessly can end up spending more on the switching than on the computation itself. Well-designed systems fuse adjacent linear operators together, push as many consecutive blocks as possible into a single HE evaluation, and drop into MPC or oblivious transfer only when there's genuinely no way around it.
Intel Labs ran this comparison directly with a two-layer network: once under CKKS using Microsoft's SEAL library, once under garbled circuits using TinyGarble2.0. The garbled-circuit version ran faster and used less memory, while the HE version supported non-interactive inference, so the client didn't have to stay online through the computation. Neither came out ahead across the board; each won on the metric that mattered for its own design goal. That result is a big part of why hybrid design became the default assumption in the field, rather than one option among several. Nobody argues much anymore about whether to mix primitives; the live question is how to shrink the cost of the boundary between them.
How the number of parties shapes what guarantees a protocol can offer
The number of computing parties in a protocol isn't a minor implementation detail. It decides what security guarantees are even on the table, and that decision ripples through everything downstream of it.
Two-party computation (2PC) is the minimal setup: a client holding the input, a server holding the model, no third party in the room. Gazelle, presented at USENIX Security in 2018, and Cheetah, which followed at USENIX Security in 2022, are the landmark systems here, and both hit strong efficiency numbers. The tradeoff is that the two parties stay mutually distrustful the whole time, with nobody around to catch cheating or break a tie.
Three-party computation (3PC), usually built on 2-out-of-3 replicated secret sharing, has become the dominant setup for Transformer inference specifically. ABY3, from CCS 2018, laid down the mixed-protocol framework that much of this later work builds on, and both PUMA and SecFormer run in this three-party model. PUMA's own architecture makes the underlying assumption almost physical: it runs its three non-colluding parties on different cloud vendors, say AWS, Google Cloud, and Microsoft Azure, so the corporate and geographic separation between them does the actual work of enforcing non-collusion instead of leaving it as a line in a security proof.
Four-party computation (4PC) adds one more participant to buy something the smaller settings can't offer as cheaply: active security. SWIFT, from USENIX Security 2021, and Tetrad both tolerate one malicious participant among the four without breaking the correctness of the final output. That's a real upgrade in guarantee, and it doesn't come free. The security-models section below unpacks that tradeoff in more depth.
Here's the part that's easy to gloss over: the non-collusion assumption is doing enormous work in every one of these multi-party settings, and no cryptographic proof can enforce it by itself. If two of PUMA's three cloud providers decided to compare notes, the protocol's guarantees fall apart no matter how sound the underlying math is. Real deployments have to treat non-collusion as an operational commitment, backed by contracts and infrastructure choices, not as a hypothesis sitting quietly inside a theorem.
What benchmark systems reveal about where performance has arrived and where it still falls short
On the vision side, CryptGPU showed private inference over large CNN architectures on ImageNet in the range of tens of seconds, with smaller networks dropping to low single-digit seconds. That's close to a usability threshold for a range of real applications, and it puts the vision case closer to solved than the language case, by a wide margin.
Language models are the harder frontier, and it isn't close. PUMA's evaluation of LLaMA-7B, published in 2023 through the SecretFlow-SPU project, generated one token in around five minutes under full MPC protection. This was the first time a model at that parameter scale had been run this way, and it came in roughly twice as fast as its predecessor, MPCFormer. Just as notably, PUMA matched plaintext inference accuracy without any fine-tuning, something earlier systems had struggled with. Holding accuracy under these constraints turns out to be its own engineering problem, separate from the fight over latency.
Microsoft Research's SIGMA pushed further, using function secret sharing to handle Softmax, GeLU, and SiLU directly, scaling up to LLaMA2-7B and LLaMA2-13B with inference times in the tens of seconds. That's real compression relative to PUMA's five-minute figure, though the two systems aren't measuring identical things exactly, so treat the comparison as a rough sense of trajectory rather than an apples-to-apples number.
By early 2025, work presented at the PPML Workshop combined FSS with GPU acceleration to get sub-second latency on ImageNet-scale inference and under a minute for LLaMA2-13B. That's the current frontier as far as the public literature shows. GPU acceleration isn't a free win everywhere, though: PIGEON, from 2024, found that GPU acceleration for non-linear layers can actually hurt performance here, because MPC circuits introduce memory overhead and constant GPU-to-CPU data shuffling that eats whatever speedup the GPU would otherwise deliver. Intuitions from plaintext ML performance engineering just don't carry over cleanly.
PermLLM, also from 2024, chases something different from raw benchmark speed: interactive latency, a few seconds per response, under realistic wide-area network conditions instead of a lab's local network. That distinction between lab conditions and real network conditions comes back harder in the communication section below.
Why non-linear layers are the central engineering problem in private Transformer inference
Softmax, GeLU, and LayerNorm are baked into the Transformer architecture. You can't train the model without them, and you can't skip them at inference time either. That would be a minor annoyance if these functions were cheap to compute securely, but they aren't. Plain linear MPC can't touch them directly, and they resist efficient polynomial approximation once you need any real precision. That's exactly why they sit at the center of nearly every performance problem in this field.
Three strategies get used to deal with this, and current systems tend to mix all three rather than pick one and stop.
First, approximation and substitution: swap Softmax or GeLU for an MPC-friendlier quadratic function, or some other simplified stand-in. This can cut costs by an order of magnitude, but it usually requires retraining the model around the substitution, and it can degrade accuracy on edge-case inputs, the kind that show up rarely in training but matter a lot when they do appear.
Second, protocol-level optimization using function secret sharing for comparison operations specifically, the building block underneath things like ReLU and max-pooling. FSS-based comparison protocols can finish a secure comparison in just two communication rounds, against the many rounds garbled circuits typically need for the same operation. MSPI-FSS shows how large that gap can get, with dramatic runtime reductions over Muse on standard image classification benchmarks.
Third, mixed-precision and ring-size reduction. Linear layers need large integer rings to avoid numeric overflow as values pile up across a matrix multiplication, but non-linear activations tend to have naturally bounded inputs and can run comfortably in smaller rings at lower precision. Communication cost scales with ring size, so matching precision to what each layer actually needs, instead of running one large ring everywhere out of caution, cuts bandwidth proportionally.
None of this makes the underlying tension go away. Push approximation too hard and you close the latency gap while opening a new one against plaintext accuracy. PUMA's result, holding accuracy steady without fine-tuning, shows the tension can be managed carefully; it doesn't show every approximation scheme will manage it equally well. That gap between one success story and a general rule is worth sitting with, because a lot of the excitement around these results glosses right over it.
Communication overhead as the binding constraint on real-world deployment
Compute has largely stopped being the bottleneck in most of these systems, and the network has taken its place.
Every MPC protocol step requires parties to exchange intermediate results, and in a deep network with dozens of layers, those exchanges pile up fast. At the bandwidths you'd actually see across a wide-area network, connecting servers in different data centers or different cloud providers, communication time can eat up most of the total inference time for a large language model. BumbleBee, presented at NDSS 2025, puts a hard number on how bad this gets: even after optimization, a single inference pass on a large BERT-variant model generates tens of gigabytes of network traffic. That's a deployment blocker for a lot of use cases, full stop.
Characterizations from 2025 literature put the gap between fully cryptographically protected LLM inference and plaintext inference at somewhere around 10,000 times slower. That's the real engineering target the field is working against, and no single trick closes a gap that size, since it takes a grind across many layers of the stack.
The mitigation toolkit stays fairly consistent across systems. Packing and SIMD operations spread ciphertext overhead across many values at once. Dimension reduction shrinks what needs to be sent in the first place. Selective outsourcing keeps only the genuinely sensitive part of a computation under MPC while the rest runs in the clear. FSS offline phases shift as much communication as possible to before the latency-sensitive query even arrives.
One distinction trips up a lot of benchmark comparisons: LAN versus WAN. Plenty of published results assume a high-bandwidth local network connecting the parties, a reasonable setup for a research paper but not necessarily for a production deployment spanning multiple cloud providers or geographies. Systems like PermLLM that explicitly target wide-area conditions are solving a genuinely harder problem than systems benchmarked purely on a local network, and that difference should shape how you read any comparative claim about how fast a given protocol runs.
Semi-honest versus malicious security models and what that choice costs
Most of the high-performance systems discussed so far, including CrypTen, PUMA, and SIGMA, run under what's called a semi-honest security model. Parties are trusted to follow the protocol as written; the worry is that they might learn something from what they legitimately observe during the computation, not that they'll actively cheat.
A malicious security model assumes something stronger and uglier: a party might deviate from the protocol on purpose, sending malformed messages or lying about intermediate values, trying to corrupt the output or pull out information it shouldn't have. That's obviously the more realistic threat model for a lot of real-world deployments, but historically it's come at a steep cost, often multiplying communication overhead well past what a semi-honest protocol needs.
That gap between the two models stays an open problem, flagged as such in the field's own survey literature on private Transformer inference, and nobody quietly solved it while everyone watched latency numbers instead. MSPI-FSS offers one path toward narrowing it, working message authentication codes into an FSS-based protocol so the system can catch malicious deviations without paying the full price of classical malicious-secure garbled circuits.
For deployers in healthcare or finance, this isn't an abstract cryptography debate. Depending on the actual threat model, semi-honest guarantees might simply not cut it, particularly if one of the computing parties has both the incentive and the technical means to deviate quietly. Four-party protocols like SWIFT and Tetrad offer one architectural route to stronger guarantees, using the extra party specifically to enable error detection and abort, spreading trust more broadly across the system instead of stacking the whole burden of honesty on two or three participants.
The framework ecosystem that makes SMPC accessible beyond protocol researchers
None of the protocol design above matters much if nobody outside a handful of cryptography labs can actually use it. This is where the open-source framework ecosystem carries a lot of the weight, and not much of the credit.
MP-SPDZ, presented by Keller at CCS 2020, covers the broadest range of protocols and adversary models of any framework in this space, which makes it the go-to for benchmarking and research comparisons. It also has a steep learning curve, and it wasn't built with production deployment in mind. CrypTen, from Meta AI and presented at NeurIPS 2021, takes a different approach: built on PyTorch and aimed at ML researchers rather than cryptographers, with an easy-to-use API that trades some flexibility for accessibility. It's limited to semi-honest security and runs into memory limits on very large models. SecretFlow-SPU, from Ant Group and presented at USENIX ATC 2023, is the general-purpose framework that hosts PUMA, and it was built for production-scale use across multiple cloud environments from day one.
Which framework a team picks quietly decides which protocol families are even available to them. A team building on CrypTen operates in a semi-honest, two-or-three-party world by default. MP-SPDZ exposes malicious-secure options, but using them well takes a level of cryptographic know-how most ML engineering teams just don't have sitting on staff.
Even the most mature of these three frameworks sits somewhere between research-grade and early production. Key management, high availability, auditability: these are the unglamorous engineering problems wedged between "the protocol works in a paper" and "the protocol runs a business's inference workload," and none of the three frameworks solves them out of the box. For teams that need private inference without building this cryptographic plumbing from scratch, purpose-built privacy-preserving AI services are an alternative worth looking at. Confidant AI, a privacy-preserving AI assistant built so user data is never collected or monetized, is one concrete instance of that approach, embedding the guarantees inside the service itself and cutting down what a customer needs to assemble on top of framework primitives.
How SMPC composes with differential privacy, TEEs, and split learning to cover gaps each technique leaves alone
SMPC isn't the only tool here, and it's worth being clear about where it stops and where other techniques pick up the slack. Stacking every available privacy technique onto a system by default is usually the wrong instinct, and it usually costs more than it buys.
Trusted execution environments, hardware features like Intel SGX or AMD SEV, offer a genuinely different threat model. Instead of spreading trust cryptographically across multiple parties, a TEE asks you to trust the hardware vendor and the isolation guarantees baked into the chip. TEEs run faster than MPC for comparable workloads a lot of the time, but they've had a rough track record with side-channel attacks, and they mean trusting a hardware supply chain most organizations don't control.
Differential privacy takes yet another angle: add calibrated statistical noise to outputs or gradients so no individual's data can be reliably pulled back out of what gets released. This runs genuinely orthogonal to SMPC rather than competing with it. DP bounds what can be learned from a result after the fact; MPC protects what gets revealed during the computation itself. A system can use both without redundancy, because they're answering different questions.
Split learning splits a model's layers between a client and a server, cutting down data exposure compared to sending raw inputs over the wire. It doesn't close the gap entirely, though, since the intermediate activations crossing the client-server boundary still leak information about the input, sometimes more than people expect, which is part of why applying MPC specifically at that split boundary has become a common way to close the remaining gap without running the entire model under full cryptographic protection.
Combining these techniques multiplies engineering complexity, and often multiplies performance cost too. The right design principle is to apply each tool where it closes a real gap in your actual threat model, and resist layering on everything out of caution. SMPC protects data during computation, TEEs handle single-party isolation, DP handles output-level privacy, and a well-designed system picks based on what it's actually defending against, not on which technique happens to be trending this year.
Where the practical frontier stands and what the remaining gap means for deployers today
Private inference over CNN-scale vision models sits close to genuinely practical today, with latencies in the low single digits of seconds for smaller networks and tens of seconds for larger ImageNet-scale architectures. That's usable for a real product, not just a research demo.
Language models tell a messier story. BERT-scale and even LLaMA-scale inference under full MPC protection has gone from theoretically possible to running in tens of seconds on frontier systems like SIGMA, and under a minute for LLaMA2-13B on the most recent GPU-accelerated FSS systems as of early 2025. That's real progress, though it's still a long way from the sub-second responsiveness people expect from a hosted LLM product, and the roughly 10,000x gap against plaintext inference won't close through any single optimization. It closes, if it closes, through the accumulation of exactly the kind of layer-by-layer engineering walked through above: better handling of non-linear layers, smarter ring-size choices, tighter HE-MPC seams, communication patterns built around real wide-area networks instead of lab conditions.
For anyone deploying private inference today, vision workloads and moderate-scale language models sit within reach now, given the right framework and the right protocol choice for your actual trust model. Frontier-scale language models under full cryptographic protection are close, measured in tens of seconds rather than minutes, but not yet at the latency regulated industries or consumer products usually demand. That gap keeps narrowing, and it narrows because of specific, granular engineering decisions like the ones covered here, not because of some general wave of better hardware showing up to save the day. Anyone weighing whether to build on this now, versus waiting another cycle, should weigh that trajectory against their own latency and compliance requirements, and be honest about which one is actually the binding constraint. There's real work still sitting on the table here, quite a bit of it, if I'm being straight about it.
