Multi-Party Inference with Confidential Computing

A hospital network wants to run patient records through a cloud-hosted large language model to flag drug interactions or summarize a case history. Two confidentiality demands collide the instant that query leaves the hospital's network. The hospital needs the records to stay unreadable to the cloud provider, to the company running the model, and to anyone else with access to the infrastructure in between. The company running the model needs its weights, the product of enormous training spend, to stay unreadable to the hospital. Ordinary encryption can protect each secret on its own. It has no answer for protecting both at the same moment, inside the same computation, which is what inference actually requires.

A comparative survey of privacy-preserving LLM inference gives these two obligations names: content privacy, under which only the client can read the prompt and the completion from end to end, and model privacy, under which the provider never exposes weights, intermediate activations, or anything that would reveal proprietary or legally protected training data. Red Hat's analysis of the problem makes the provider's side concrete: a frontier model reflects a massive capital investment and counts as real intellectual property, and running it remotely without protection leaves it open to inspection, copying, or outright theft by anyone with privileged access to the machine it runs on, including the cloud operator's own staff.

Running the model entirely inside the client's own data center looks like an obvious fix, but it only protects the hospital's records while leaving the provider's weights exposed on the client's infrastructure. Shifting the physical location of the computation does not resolve the underlying conflict. A provider protecting its weights cannot simply give the model to the client, and a client protecting its data cannot simply give its records to the provider. Both parties need to compute something together without either one trusting the other with their respective secret, and that requirement is what the rest of this piece works through.

The processing gap: data in use versus data at rest or in transit

Encrypting a file sitting on a disk is a solved problem. What breaks down is the moment in between: the point where a computation actually needs to read the data to do anything useful with it. Standard processors, the CPUs and GPUs that run nearly all cloud workloads, operate on plaintext. They cannot add two numbers, multiply a matrix, or run a transformer layer on ciphertext directly. So any classical system that wants to compute on encrypted data has to decrypt it first, in memory, where the operating system, the hypervisor, or anyone else with sufficient privilege on that machine can read it.

The Confidential Computing Consortium's IDC white paper describes this as the third stage in the evolution of data security, arriving after encryption at rest and encryption in motion were already standard practice. The paper notes that protocols like TLS protect data while it travels, but the encryption gets stripped away during processing, which can leave the data sitting exposed at the exact moment it matters most.

Cloud infrastructure does not cause this gap, but it does multiply who can see into it. A single on-premises server has one operator with that level of access. Patching around this with better access controls or stricter logging does not close the gap, because the gap is structural: plaintext has to exist somewhere for arithmetic to happen. Closing it requires a different kind of hardware, one built to make that moment of exposure impossible rather than just monitored.

How TEEs create a hardware-enforced boundary around computation

Trusted Execution Environments answer the in-use problem by building a hardware boundary around the computation itself. Inside that boundary, called an enclave, code runs and data gets decrypted, but nothing outside the enclave, not the cloud operator, not the host operating system, not another tenant on the same physical machine, can read what's happening inside it. The plaintext exists only within a boundary the hardware itself enforces and polices.

This isolation is built into specific chip technologies already deployed at scale: Intel SGX, AMD SEV, and AWS Nitro Enclaves each implement their own version of it, and all three are designed so that data and the code operating on it stay protected even while that code is actively running, not just before or after.

Red Hat's engineering work on this describes the enclave in plain terms: a digital vault built directly into the CPU and GPU. Model weights get loaded into that vault only after the environment has been cryptographically checked and confirmed to be what it claims to be. Nothing goes in until the hardware has proven itself.

The model provider's half of the problem gets solved through a specific mechanism: Red Hat's approach encrypts the entire container image that carries the model weights, so the file stays an unreadable cryptographic payload the whole time it travels from the provider's systems to the cloud, and only becomes readable once it actually arrives inside a verified enclave. The weights are protected in transit, at rest, and up until the final moment they're needed, at which point they're protected by the enclave boundary instead.

Put together, this lets a provider run a proprietary model on a client's sensitive data without the client ever seeing the model's weights, and without the provider ever seeing the client's plaintext inputs. One enclave, one hardware boundary, mediates both secrets at once.

Attestation: how mutually distrusting parties verify the enclave before sharing secrets

An enclave by itself does not earn anyone's trust automatically. Before the hospital submits patient records and before the model provider releases its weights, both need a way to check that the enclave they're about to trust is genuine, has not been tampered with, and is running precisely the software it claims to run, not a modified version someone slipped in to quietly exfiltrate data. That check is remote attestation.

Red Hat's implementation of this calls it mutual attestation, and it runs in two directions at once. The model provider checks that the TEE is authentic and uncompromised before handing over the private key needed to decrypt the container image holding the weights, which confirms the intellectual property will only ever execute inside an environment meeting the security level the provider specified. The end user, separately, checks that the TEE is configured to run exactly the approved software, the specific LLM and its inference code, confirming their sensitive inputs will be processed according to the policy they agreed to and nothing else.

Attestation is what lets two organizations that trust neither each other nor each other's cloud providers enter a shared computation without needing some third party to vouch for everyone. The hardware itself does the job a trusted intermediary would otherwise have to do, acting as a kind of notary that both sides can independently verify.

The πCreds system, built by researchers at Cornell Tech, MIT, and UCSD, shows attestation working in a live application rather than as an abstract guarantee. The system runs a TEE-protected pipeline that pulls in authenticated user data from outside sources and runs LLM inference over it inside the enclave. The attestation record generated by that process lets a credential be issued proving the inference actually happened correctly, without revealing the underlying data that went in or the model's internal reasoning that produced the result. That is attestation functioning across multiple distrusting parties at once: the user supplying data, the sources authenticating it, and whoever relies on the resulting credential downstream.

What MPC adds beyond hardware trust

Secure multi-party computation takes a different route to the same destination. Instead of relying on a hardware boundary that someone has to trust was built and verified correctly, MPC splits both the user's inputs and the model's weights into secret shares, distributed across several parties who have agreed not to collude. Those parties jointly compute the inference result through cryptographic protocols, and at no point does any single one of them ever hold a full copy of the plaintext.

The comparative survey frames MPC as enabling joint evaluation over these secret-shared inputs, and draws a sharp line against TEEs on the question of what you actually have to trust: MPC's guarantees come from mathematics, not from faith in a chip manufacturer or confidence that a processor's internal microarchitecture has no exploitable flaws.

Recent research has started chipping away at that cost. Work using packed Shamir secret sharing for deep learning has produced real reductions in offline, online, and total communication overhead compared to earlier scalable MPC protocols, tested on networks as deep as VGG16. Separate work on dealer-assisted multi-party secure comparison targets the preprocessing cost behind those comparison operations specifically, producing speedups over prior MPC frameworks by generating richer correlated randomness ahead of time rather than computing it on the fly.

MPC's guarantee holds up even if the hardware running it has been compromised, as long as the parties involved genuinely do not collude, a different and, at present, slower and more communication-heavy promise than a TEE makes. Choosing between the two comes down to which set of assumptions, hardware integrity or non-collusion among parties, a given deployment is more willing to stake its privacy guarantee on.

Hybrid designs: TEEs combined with cryptographic protocols as the practical architecture for multi-party LLM inference

Neither approach, taken alone, covers the whole problem cleanly at the scale modern LLMs operate at. MPC carries communication overhead that grows badly as models get larger. Each one is strong exactly where the other is weak, which is the practical case for combining them rather than picking a side.

Read this way, the hybrid approach is a deliberate point along that trajectory, chosen for where the technology actually stands right now.

One version of that hybrid looks like this in practice: run the bulk of transformer inference, the linear layers, the attention mechanism, inside a TEE, where hardware isolation handles the heavy computation efficiently, and reserve MPC or homomorphic techniques for the specific non-linear operations or the exchanges between parties where hardware trust is the most contested assumption in the whole system.

Red Hat's engineering work on confidential computing for AI inference points in this direction already, aiming to make TEE-based inference something cloud-native systems can use without heavy custom engineering. The container-image encryption and mutual attestation model it has built functions as a skeleton, a structure cryptographic augmentation can be layered onto as those techniques mature rather than something that has to be torn out and replaced.

The πCreds system again serves as a working example of this blend rather than a pure instance of either approach. Neither the hardware isolation nor the cryptographic verification is doing the whole job alone.

The side-channel objection to TEE-based inference

The strongest challenge to TEE-based confidential inference has little to do with speed or cost. It concerns what the enclave boundary actually holds up against. Microarchitectural features inside the chip itself, including cache access patterns, branch prediction, and speculative execution, have been used to pull cryptographic keys and sensitive data out of enclaves that were supposed to keep them sealed off. These are side-channel attacks: they work by watching how the hardware behaves rather than by breaking through the isolation boundary directly. They can succeed even when the TEE is functioning exactly as designed.

The Confidential Computing Consortium's own technical analysis does not shy away from this. It states that TEEs provide assurance relative to their stated threat model, and that assurance depends on an assumption: that there are no exploitable side-channels available to the system's owner or to other parties with access to the machine, which could be used to infer something about the data or the computation running inside.

Framed in starker terms, the objection becomes a question about what kind of guarantee is acceptable. MPC's guarantees are cryptographic and composable, holding regardless of what hardware they run on. Choosing a TEE instead means accepting a performance advantage in exchange for a guarantee that rests, in part, on trusting a chip's physical construction. That tradeoff is also a judgment call about how much confidence to place in hardware manufacturers and in the completeness of their own side-channel defenses.

The field's response to this has not been to abandon TEEs. It has been to treat the objection as the reason hybrid and crypto-augmented designs exist in the first place: push the operations most exposed to side-channel analysis into MPC or verifiable computation, and keep the bulk of the computation inside the TEE, where hardware isolation is adequate and the overhead of running everything through full MPC would be prohibitive. The side-channel risk does not get eliminated by this split. It gets concentrated into a smaller portion of the system, where the cost of applying a stronger cryptographic guarantee is easier to justify.

Multi-party confidential inference in practice today

The organizations with the most to gain from multi-party confidential inference are the ones already sitting on data they are legally or contractually barred from sharing in plaintext: hospitals and pharmaceutical companies bound by patient privacy law, banks and insurers bound by financial regulation.

Making that work requires more than installing a TEE and calling it done. None of these pieces is optional dressing on top of the core design. They are what turns a hardware enclave sitting in a cloud data center into something a hospital's compliance office and a model provider's legal team can both sign off on, the actual precondition for this kind of collaboration. The same architectural logic applies at the consumer end of the AI market: Confidant AI, a privacy-preserving AI assistant built so the service operator itself cannot read users' prompts, is one example of that design philosophy carried into a general-purpose product.