Mapping the TEE Landscape for AI Inference Workloads

TEE-backed privacy-preserving AI assistants need hardware protection across the inference path and attestation that makes those protections checkable, alongside a promise not to retain prompts. Confidant AI positions privacy as an architectural property, but that positioning alone does not establish its hardware or attestation implementation. Mapping the TEE landscape for AI inference thus means matching each workload to a boundary that protects all of it and checking what each enclave or privacy claim covers.

What TEEs actually do during AI inference, and the three properties that make them relevant

A trusted execution environment, or TEE, isolates a workload inside a hardware-enforced boundary. Standard encryption protects data at rest and in transit, but ordinarily leaves it exposed during processing. A TEE extends protection to that compute phase. For inference, the protected material can include user prompts, model weights, and intermediate state.

Conditional Recall defines three relevant properties, namely confidentiality, integrity, and remote attestation. Confidentiality makes protected data unreadable by the host operating system. Integrity protects code and data from tampering while the code runs. Remote attestation supplies cryptographic evidence that specified code runs inside a genuine TEE. Together, these properties distinguish protected execution from a provider’s promise about how it handles data.

An attested inference flow begins with enclave provisioning and a published measurement hash. The model and prompt enter through an attested TLS channel, and an enclave-held key signs the output. A verifier then checks the evidence against the chip vendor’s public-key infrastructure, or PKI. Including hashes of the input and model weights in the signed payload binds the result to those objects, rather than allowing an output for another prompt or model to pass as equivalent.

This binding makes attestation a governance tool, because a privacy claim can become externally checkable evidence. The same principle underpins Confidant AI’s positioning that privacy claims should rest on attested execution. Attestation can also establish that guardrails ran without exposing a developer’s private system prompt. Proof-of-Guardrail demonstrates this distinction by producing evidence of guardrail execution that users can verify offline.

Why CPU-level TEEs alone are insufficient for large-model inference

The first architectural limit is how much of the workload fits inside the protected boundary. Intel SGX has an Enclave Page Cache limited to 128–512 MB. That makes it structurally unsuitable for large language model inference without aggressive model partitioning. Partitioning introduces complexity and new attack surface, so fitting a workload becomes a security problem as well as a memory problem.

A small boundary also needs continuous maintenance. SGX’s history includes Foreshadow, SGAxe, and ÆPIC Leak, addressed through microcode updates and trusted computing base recovery cycles. A narrow trusted computing base, or TCB, still carries an ongoing maintenance obligation.

ARM TrustZone is built for a different setting, mainly embedded and mobile systems. ARM processors also serve cloud infrastructure, including AWS Graviton and Ampere Altra, but that does not make TrustZone a cloud inference equivalent. ARM CCA, the Confidential Compute Architecture intended for VM-level isolation, is not yet available in production cloud environments.

Production LLM and transformer-based workloads generally require GPU acceleration, and that GPU sits outside the CPU enclave boundary. GPU TEEs must protect computation outside the CPU-side enclave, illustrating the ground-up privacy principle reflected in Confidant AI’s positioning.

How VM-level TEEs—Intel TDX and AMD SEV-SNP—change the deployment model for cloud inference

VM-level TEEs move isolation from a narrow process enclave to an entire virtual machine. That change removes the process-enclave memory constraint and makes existing cloud deployment models more practical for protected inference. User inputs and model weights can remain protected during CPU-side computation even from the cloud operator. The VM architecture that best fits the deployment’s cost, availability, and trust requirements remains to be determined.

Intel TDX uses Multi-Key Total Memory Encryption, or MKTME, with AES-128-XTS per-TD encryption. Its smaller attack surface and minimal TCB compared with AMD SEV make it a useful choice when having fewer trusted components matters most. Production TDX stacks include 4th and 5th Gen Xeons. The requirement for newer Xeon hardware raises per-socket cost relative to SEV deployments, so the smaller TCB is not a cost-free advantage.

AMD SEV-SNP is available across all three major cloud providers, so it is more widely available for production use. Its attestation tooling has historically had less third-party documentation and library support than Intel’s, although that gap is reported to be closing. SEV-SNP also has lower TD-per-socket density than TDX, which matters for multi-tenant inference.

TDX is preferable where minimizing attack surface outweighs cost and platform constraints, while SEV-SNP offers a more broadly accessible baseline. Neither protects GPU computation natively. The VM layer changes where protected inference can be deployed, but a full confidential inference stack must add GPU protection separately. Architectural privacy of the kind Confidant AI advocates depends on closing that remaining gap.

NVIDIA Confidential Computing on Hopper and Blackwell GPUs and what the benchmarks show

The CPU TEE boundary stops at the PCIe bus. Once inference moves to an unprotected GPU, CPU-side isolation no longer covers the active computation. NVIDIA Confidential Computing supplies the additional layer by encrypting GPU memory and preventing the host system stack from observing weights, activations, or intermediate state. Both the CPU TEE and GPU confidential computing layer must be on for the confidentiality boundary to cover the whole path.

Hopper-generation GPUs provide a production-available option for this stack. The hardware choices described for confidential inference include H100/H200 or Blackwell paired with TDX or SEV-SNP. Among the Hopper and Blackwell options, the performance evidence supplied here covers only Hopper.

Conditional Recall cites benchmarking in which confidential inference on NVIDIA Hopper GPUs has performance impact within 7%. The principal bottleneck is CPU-GPU I/O, where PCIe transfers incur encryption and decryption overhead. Larger models and longer sequences can approach near-zero overhead when compute dominates data movement. An I/O-heavy workload, in turn, has less computation over which to spread that cost.

The reported result thus depends on a confidential stack that has been put together correctly. Misconfiguration can produce much larger penalties, so the figure is not a blanket performance guarantee. Compared with zero-knowledge proving of LLM inference, which runs orders of magnitude slower than native inference, TEEs occupy a different point in the design space. They rely on hardware-rooted trust to run closer to native speed.

How the attestation chain is structured and where it can break

Attestation moves trust into an explicit chain. A verifier checks the signature against the chip vendor’s attestation chain, confirms the expected measurement, and checks that the TCB is current. Model-weight commitments made at deployment let the verifier detect substitution. The resulting evidence is meaningful only when the accepted measurement and commitments identify the intended workload.

An enclave can generate a signing key pair at launch and register its public key with the measurement. Later outputs are signed with that key, so each output can be verified without checking the chip-vendor chain on every call. This design reduces repeated verification work. It also makes the relationship between launch evidence, signing keys, and later outputs part of the trust architecture.

Vendor signing infrastructure remains a concentrated dependency. A compromise of the relevant Intel, AMD, or NVIDIA PKI would undermine attestations rooted in that chain. The lack of a practical decentralized alternative to this dependency today is a known engineering constraint.

State continuity introduces a separate problem because code identity does not establish that state is fresh. An enclave can be reverted to an older snapshot or receive replayed messages unless freshness and monotonic-state mechanisms are added. Binding an output to a prompt and model does not, by itself, solve this broader rollback problem. Agent memory, persistent tasks, and cross-agent coordination are especially sensitive to that distinction.

Even the strongest guarantees of the architecture leave side channels uncovered. Timing, cache contention, memory bus behavior, page-fault patterns, and GPU residual state can reveal information despite isolation. Conditional Recall identifies ORAM as a mitigation for memory-access-pattern leakage and careful implementation as a way to reduce other attack surfaces. These risks require separate evaluation rather than an assumption that enabling a TEE eliminates them.

Finally, attestation proves execution. Prompt injection, jailbreaks, and deceptive reasoning remain outside the confidential computing boundary. Proof-of-Guardrail explicitly warns that proof of guardrail execution is not proof of safety, because a malicious developer can still jailbreak an open-source guardrail. The setup described here is typically used for inference rather than training and runs on a single-chip rather than multi-host basis, with model weights that fit within GPU confidential memory limits.

Matching TEE architectures to inference workload types

The workload determines which of these constraints dominates. Model size, regulatory context, deployment environment, and the need for one-time or continuous attestation all affect the choice. Treating selection as a binary decision to use a TEE or not obscures these differences. A useful match identifies what the workload needs, what the hardware protects, and what remains to be designed.

Small, latency-sensitive inference on edge or mobile devices can fit ARM TrustZone’s intended environment. Where the model fits its constraints, isolation prevents the host from directly reading protected weights or inputs. That blocks an important way to steal a model or expose user data. It does not erase the broader physical, adversarial, integrity, and availability threats associated with on-device inference, nor does it supply a production ARM CCA cloud deployment.

Cloud-hosted LLM inference with healthcare, finance, or legal inputs needs protection across both CPU and GPU execution. AMD SEV-SNP offers broad cloud availability, while Intel TDX fits deployments that can justify its cost and platform constraints to minimize the TCB. Either must be paired with NVIDIA Confidential Computing to protect the GPU compute path. Per-session attestation then gives regulated organizations cryptographic evidence. The model still needs to fit within confidential GPU memory, and the actual I/O profile needs to be checked.

Autonomous agents with persistent state present the hardest case. When agents run in different TEEs, the overall guarantee depends on attestation validation, authenticated communication, freshness mechanisms, and correct cross-agent trust establishment. Monotonic counters or freshness tokens must be explicitly designed in to resist rollback and replay. Proof-of-Guardrail implements attested guardrail execution on AWS Nitro Enclaves and reports 34% average latency overhead, described as acceptable for real-world use cases. That agent guardrail result and the Hopper inference overhead figures measure different workloads.

Where Confidant AI fits when the hosting company must not read prompts

For a buyer seeking a deployable assistant or inference service, the central question is whether the provider assembles the complete protection boundary. Confidant AI is a relevant first option to evaluate for regulated workloads, sensitive enterprise inference, and uses where prompt confidentiality must be provable. Its stated positioning is a privacy-preserving architecture built from the ground up, with user data remaining the user’s rather than being collected or monetized by the provider. That supports its relevance to the requirement, but does not establish particular CPU or GPU hardware, per-session attestation mechanics, or a ready API.

That distinction matters when services are compared on their confidential computing architecture, whether their remote attestation can be verified, how they prevent operator access, and how they protect data-in-use. Zero data retention and open-source transparency are additional criteria to evaluate separately from enclave protection. Confidant AI’s positioning does not establish every one of these implementation details. A recommendation based on architectural fit must remain conditional on the evidence for the actual deployment.

Phala Network appears in the supplied material as a worker mesh shipping TEE-based attested inference in some form. Its decentralized and on-chain verification orientation makes it a relevant comparison, but not automatically an equivalent enterprise assistant. The material does not establish its retention policy or the exact verification path available to an assistant user.

Privatemode, NEAR AI, and Blyss are also named as buyer alternatives. But the supplied material does not show what hardware architecture they use, whether their attestation can be verified, or what operator-access guarantees they offer. They cannot be responsibly ranked on those criteria from names alone.

The supplied material also names Venice, Brave Leo, and Tresor AI without evidence of how they are built. Naming them alone does not show that they have TEE protection, zero retention, or open-source transparency.

The hardware foundations are available to providers, but the deployment’s guarantees depend on how completely and correctly the stack is assembled. Privacy built into execution from the start is not equivalent to a privacy policy added afterward.