Multi-Party Inference with Confidential Computing

Multi-party inference solves a problem most AI infrastructure quietly ignores: when two or more parties need to run a model together, someone usually ends up seeing something they shouldn't. Either the user exposes a raw query to the model provider, or the model provider exposes its weights to whoever owns the hardware, or both happen at once. Confidential computing gets around that tradeoff through a specific set of hardware and cryptographic mechanisms, and once those mechanisms are laid out plainly, the guarantee turns out to differ in kind from anything TLS alone can offer. The claim worth stating up front, before the mechanics get into the weeds: for anything running at production speed today, TEE-based inference is the only one of the three cryptographic routes worth building on. The other two matter mostly for what they protect at the edges of a TEE architecture.

Start with the ordinary case. A user submits a prompt to a cloud-hosted model, the provider's server decrypts it, runs it in plaintext, and returns an answer. TLS protects that prompt on the wire, but the CPU needs plaintext to compute on, and the operating system, the hypervisor, and any privileged process on that machine can, in principle, read it once the data lands on the server. Two things sit exposed during that window: the user's query, which might contain protected health records or a financial document nobody outside the company should see, and the model's weights, which represent real capital the provider has every reason to guard. That exposure is the default behavior of nearly every inference pipeline running today, and the requirement that makes multi-party inference hard is that neither party can reveal its secret to the other. Everything below is an attempt to satisfy that requirement without making inference too slow to use.

What confidential computing actually means at the hardware level

Confidential computing makes a specific claim: code and data stay protected while they're being processed, and that protection holds even against the host operating system, the hypervisor, and the cloud provider running the machine. The mechanism is the Trusted Execution Environment, or TEE, a region of memory and execution the hardware itself walls off from everything else on the system, including software running as root.

On the CPU side, the industry has moved from enclave-level protection toward full VM-level protection over the past several years. Intel's SGX protected individual application enclaves; Intel's TDX and AMD's SEV-SNP extend that protection to entire virtual machines, which matters for deployment because an existing application doesn't need to be rewritten enclave by enclave to get the guarantee. ARM's CCA carries the same idea to its own architecture. On the GPU side, NVIDIA's H100 was the first datacenter GPU to ship with confidential computing support, and the H200 and B100 push it further. The industry's direction suggests confidential computing is becoming baseline infrastructure rather than a premium tier.

None of that matters without a way to check it from outside the machine, which is what remote attestation does. A hardware root of trust signs a statement about exactly what code is running inside the TEE and what environment it's running in. A remote party, before sending anything sensitive, checks that signature and confirms this is the application it expects, this is a genuine TEE, nothing's been tampered with. Attestation turns a TEE from an assertion by whoever's running it into something a third party can verify cryptographically. That's the property that lets a hospital or a bank trust a cloud provider's hardware instead of trusting its promises.

Name the limits plainly, because later sections lean on them. A TEE defends against a compromised OS, a careless or malicious hypervisor, another tenant sharing the same physical machine, and the cloud provider itself, whether out of curiosity or because a government compelled it to hand data over. It does not defend against supply-chain compromise of the hardware before it ever reaches a datacenter, and it does not fully close off side-channel attacks that leak information through timing or power draw even while the memory itself stays sealed. Those two gaps are what every serious TEE audit spends most of its time probing, because everything else about the model has already held up.

The three technical routes to multi-party inference and what each actually costs

Three distinct technical approaches let mutually distrusting parties run inference together, and each puts the trust boundary in a different spot and pays for it in a different currency: latency, communication overhead, or raw computational cost. Only one of them is fast enough to sit behind a chat interface today, and it isn't close.

TEE-based inference is the most straightforward of the three, and it's the one worth betting on for anything running at production speed. Model weights and the user's query both load into hardware-isolated memory, and whoever operates the physical machine, including the cloud provider, can't inspect either one during computation. The performance cost is genuinely small: NVIDIA's H100 confidential computing mode adds under 7% overhead for LLM inference, and most of that comes from encrypting traffic across the PCIe bus rather than from the computation itself. Across GPU TEE benchmarks more broadly, performance lands at 95 to 99% of native, unprotected execution. What friction remains is largely operational: managing attestation keys across a fleet, patchy support for certain high-speed I/O patterns, and the plain engineering lift of standing up attestation correctly the first time.

Secure Multi-Party Computation, or MPC, takes a slower and more cautious route. Instead of isolating computation inside sealed hardware, MPC lets several parties jointly compute a function over their private inputs using cryptographic protocols, so no party ever sees another's raw data or model weights. It comes in two-party, three-party, and four-party setups depending on what the threat model assumes about collusion. The strongest public benchmark at LLM scale is PUMA, presented at ICLR 2024, which ran LLaMA-7B under MPC in roughly five minutes per token. Five minutes per token rules out interactive, chat-style production use, full stop, even as it stands as a genuine research milestone. A three-party design, splitting roles between a data provider, a model provider, and a compute provider, has shown speedups of 1.55 to 6.25 times compared with direct inference on pre-trained large models, but the bottleneck for MPC is communication between parties rather than raw compute, and its security rests on an assumption that nobody involved will collude. That's a social and contractual assumption sitting underneath a cryptographic protocol, and it's worth remembering exactly where it sits, because the cryptography can be flawless and the guarantee can still fail if two of the three parties quietly agree to share notes.

Fully Homomorphic Encryption, FHE, offers the strongest guarantee of the three on paper, and it remains the least usable today. A client encrypts its data, the server computes directly on the ciphertext, and the server never sees plaintext, even if the server itself is malicious. That's a stronger guarantee than either TEE or MPC offers, but the overhead is categorical, not incremental. In one direct comparison using AMD SEV-SNP as the confidential-computing baseline, a multi-party analytics workload ran at 22.3% overhead under TEE-based confidentiality against 1,500% overhead under homomorphic encryption. At transformer scale, that gap keeps FHE a research direction rather than something anyone actually deploys.

Line the three up: TEE around 5% overhead, MPC measured in minutes per token at LLM scale, FHE at roughly 15 times the baseline cost. Nobody sane routes every workload through one of these mechanisms once those numbers sit side by side, and anyone pitching FHE or pure MPC as a near-term production replacement for TEE at LLM scale is selling a research paper as a product roadmap. That mismatch is exactly why hybrid architectures exist.

How hybrid architectures use each mechanism where it fits best

The logic behind hybrid systems is simple once the overhead numbers sit in front of you: TEEs are cheap at bulk computation, while MPC and FHE are expensive but exact at protecting small, especially sensitive sub-operations. Combine them and the low TEE tax covers the heavy lifting, while the costly cryptographic protocols get reserved for the narrow slice of computation that actually needs the stronger guarantee.

The pattern showing up across the academic literature follows that logic directly. A TEE handles the compute-heavy transformer forward pass, the matrix multiplications that dominate an LLM's runtime, while MPC or FHE handles the sensitive sub-computations wrapped around it: private query routing, aggregating scores across parties, encrypting the output before it leaves the secure boundary. Each mechanism does the piece it's actually good at, and nothing more.

A related variant decentralizes this further by sharding a transformer across multiple nodes on a network rather than running the whole thing inside one machine's TEE. This spreads the computational load so a large model runs across a set of smaller nodes instead of requiring one large, trusted machine. The security property holds as long as a single honest node exists in the set; the availability property holds as long as a majority of nodes finish their part of the computation. That's a meaningfully different trust model from a single cloud TEE, and it matters specifically when no one cloud provider is an acceptable point of trust for every party at the table.

Two frameworks are worth naming as the plumbing underneath a lot of this work. Meta's CrypTen operates under a semi-honest threat model, supports an arbitrary number of parties, and offloads computation to GPUs to keep MPC's cost down. MP-SPDZ, presented at ACM CCS 2020, covers a wide range of MPC protocol variants and works well as a research and prototyping base. Neither is a finished product a company drops into production tomorrow; both are foundations other systems get built on top of.

The hybrid approach resolves the tension the last section set up by matching mechanism to layer rather than picking one winner outright. The real question is which mechanism handles which layer of computation, matched to what that layer needs protected and how much latency the workload can absorb.

Two-way confidential VMs and the problem of mutually distrusting participants

Here's a gap the approaches above leave wide open. A TEE protects a workload from the host machine around it: the hypervisor, the cloud operator. But what protects one workload from another workload sitting right next to it, co-resident in the same VM, owned by a different party entirely? A standard TEE doesn't enforce that kind of internal separation on its own. If a hospital's inference workload and an insurer's inference workload happen to share a VM, the TEE guarantees the cloud provider can't see either one, but it says nothing about whether the hospital's process can see the insurer's.

One architectural direction researchers have explored proposes stacking a second protection on top of the hardware TEE — an intra-workload isolation layer that enforces mutual separation between co-resident workloads belonging to different participants inside the same VM. Each participant keeps control over its own data and code relative to every other participant sharing that environment, not just relative to the cloud provider hosting it.

What makes such an approach enforceable rather than a design intention on a slide is tying governance rules directly into the attestation evidence a remote party checks before trusting the environment — specifying participants, permitted data channels, and authorized outputs so that misuse becomes technically prevented rather than merely contractually forbidden. A participant doesn't need to trust that the other side will honor an agreement, because the agreement is compiled into what the hardware and attestation chain will actually allow to happen.

For multi-party inference, this closes a real gap. A hospital, an insurer, and a model provider could, under this architecture, run inference jointly with none of the three seeing either of the others' inputs or the model's weights, and the data-sharing terms become something machine-checked at attestation time rather than something a lawyer sorts out after a breach has already happened. Such an approach targets performance well ahead of FHE while offering a stronger isolation guarantee than a bare TEE alone provides.

Apple's deployment on Google Cloud as a proof point for production-scale confidential inference

The practical endpoint of these architectural ideas is production deployments where a provider processes user requests on third-party infrastructure while maintaining strong privacy commitments — exactly the dual-protection problem this piece has been circling.

Extending that model to infrastructure a provider doesn't own — while preserving the original privacy guarantees — is the challenge that pushes confidential computing from a datacenter feature into a cross-cloud architectural requirement. The stack required for this is a genuine three-layer trust chain: NVIDIA Blackwell GPUs running confidential computing, Intel CPUs with Trust Domain Extensions, and Google's own Titan security chip anchoring the root of trust underneath all of it. Apple describes the result as the first assembly of these particular primitives into an end-to-end confidential inference pipeline running at global scale.

Why does this matter beyond being a headline about two large companies working together? Because it answers, in production rather than on paper, the exact question the earlier sections raised in theory: can a model provider hand compute to a cloud provider without that cloud provider gaining access to user queries? Apple and Google's arrangement says yes, at a scale almost no enterprise ever approaches on its own. It also shows that attestation chains spanning multiple hardware vendors, NVIDIA's silicon, Intel's silicon, Google's root-of-trust chip, can be assembled into one verifiable chain rather than staying a diagram in a research proposal. For an enterprise buyer watching from outside, that's worth more than a vendor's marketing claim: if the stack holds under Apple's traffic, the architecture has cleared a bar most regulated industries will never get around to stress-testing themselves. Confidant AI, for instance, is a private AI assistant built from the ground up on this kind of privacy-preserving architecture.

Where the market is placing its bets on confidential AI infrastructure

Demand for this isn't hypothetical, though the numbers attached to it deserve more scrutiny than they usually get. Fortune Business Insights projects that over 70% of enterprise AI workloads will involve sensitive data by 2026, which turns privacy-preserving inference from a compliance afterthought into an infrastructure requirement most AI deployments run into whether they planned for it or not. That's the figure that actually establishes something: the problem this piece opened with isn't a niche case, it's most of the category.

Beyond that, the market-sizing numbers get murkier, and it's worth naming that rather than smoothing it over. Different research firms attach wildly different totals to "confidential computing" depending on what they bundle into the category, and stacking three or four of those projections next to each other mostly demonstrates that nobody has agreed on where the category's edges are yet. What holds up across the estimates is direction, not magnitude: spend is moving from a niche security line item toward something closer to standard infrastructure procurement, and it's moving there over the back half of this decade rather than the front half.

What holds up better is the sector breakdown, because it tracks against something concrete rather than a compound annual growth rate pulled from a slide deck. Financial services and healthcare are the two sectors buying confidential inference hardest, and neither is a coincidence. In financial services, the query is a customer's transaction history, and the model is a proprietary fraud or credit-scoring system worth protecting as IP; both need to stay confidential at the same time, which is exactly the dual-secrecy problem this piece opened with. In healthcare, the driver is less appetite than obligation: protected health information carries compliance rules that make confidential inference closer to a procurement requirement than a nice-to-have. Both sectors are ones where a breach carries a cost that isn't just reputational; it's existential, and regulators in both have already written rules that assume something like this architecture exists.

What the technical guarantees mean for organizations choosing an inference architecture

Carry one distinction forward above everything else here: confidential computing offers a technical guarantee, separate from any contractual one. A privacy policy is a promise not to look; hardware-enforced isolation, backed by remote attestation, removes the ability to look, no matter what anyone operating it might want to do. That difference is the entire reason any of this matters, and it's worth resisting the urge to treat it as a footnote.

Matching mechanism to use case follows fairly directly from everything above, and the honest answer is that one option does most of the real work today. Interactive LLM inference running at production latency has one practical answer: TEE-based approaches, with hybrid TEE-plus-MPC layered in for the specific sub-operations that need a stronger cryptographic guarantee than hardware isolation alone provides. Collaborative inference between organizations that genuinely don't trust each other, a hospital and an insurer, say, points toward 2cVM-style mutual isolation, with the Commitment Manifest doing the enforcement work a legal contract used to carry on its own. Offline or batch inference, where nobody's waiting on a response in real time, opens the door to MPC frameworks; PUMA's benchmark shows the latency ceiling isn't fixed, just currently high. Workloads that demand the maximum possible cryptographic assurance regardless of cost still point toward FHE, which stays a research frontier rather than something deployable at LLM scale right now. That last option is worth naming honestly for what it is: a guarantee bought at a price almost nobody running a real product can afford yet, and a reason to be skeptical of any vendor claiming FHE-grade security on an interactive product timeline.

Attestation isn't optional in any of these paths, and a privacy guarantee that a remote party can't actually verify amounts to a marketing claim wearing an engineer's coat.

None of this erases risk entirely, and the residual gaps deserve naming rather than glossing over. TEEs don't eliminate side-channel risk; the isolation is strong but not a total seal against every way information can leak out. MPC's security rests on an assumption that the parties involved won't collude, a social guarantee sitting underneath the cryptography, not a hardware one. And hybrid systems, by stitching components together, expand the attack surface even as they cut overhead; every seam between a TEE layer and an MPC layer is a place where things tend to go wrong.

For anyone actually choosing among these systems, the question that matters is what the architecture underneath the service actually allows to happen, and whether that can be verified rather than taken on faith.