NVIDIA H100 Confidential Computing Mode Architecture

Data sitting on a disk can be encrypted. But data loaded into a GPU's memory while the GPU is actually working on it has, until recently, had no such protection at all. TLS and disk encryption solve real problems, but neither one touches the moment a model's weights and a user's inputs sit in VRAM during computation, because that problem sits outside what either technology was built to do.

On a GPU without confidential computing, VRAM holds plaintext the entire time a kernel runs, and the hypervisor managing that GPU has direct memory access to its physical address space. A compromised management layer, a live-migration operation moving a VM between physical hosts, or simply a privileged insider with the right credentials can pull model weights and inference inputs straight out of high-bandwidth memory using memory-scanning tools that are publicly available, with no custom exploit required. The same exposure extends to the PCIe bus itself: traffic moving between CPU and GPU travels unencrypted on non-CC hardware, giving anyone with physical or privileged access to the machine a second place to intercept the same data.

This matters most acutely on shared infrastructure. Add to this a narrower but genuine risk: cold-boot attacks against high-bandwidth memory, which remain a real concern in colocation facilities and high-security data centers even though the conditions for pulling one off are constrained.

None of this was solved by confidential computing as it existed before Hopper. CPU-based confidential computing already protected data inside CPU enclaves, but that protection was built for workloads far less demanding than AI at scale, and it stopped being useful the moment a workload needed serious GPU compute. Trying to stretch a CPU trusted execution environment to cover GPU work meant passing data back out across the PCIe bus, the very boundary that travels in the clear, which undid the protection the CPU enclave was supposed to provide. The gap was never a matter of tightening configuration. It called for a change at the hardware level, because no software policy can encrypt memory that the silicon itself exposes as plaintext.

The H100 as the first GPU meeting the formal trusted execution environment definition

The H100 is the first GPU built to meet the Confidential Computing Consortium's formal definition of a trusted execution environment, and it does so by anchoring that environment in a hardware root of trust fixed directly on the die. Clearing that bar required engineering work that reached well below the application layer.

Meeting this definition required NVIDIA to create new secure firmware and microcode, enable confidential-computing-capable paths in the CUDA driver, and establish attestation verification flows, none of which existed before Hopper. Volta introduced AES authentication for firmware, and Turing and Ampere layered on encrypted firmware along with revocation support, but none of them carried an on-die root of trust or an attested boot sequence. Those protections mattered, but they addressed firmware integrity in isolation rather than building a verifiable chain running from power-on to workload execution.

The on-die root of trust is what makes everything described in the rest of this piece possible. Without a hardware anchor fixed at manufacture, boot measurements have nothing to anchor their trust in, and an attestation report has nothing credible underneath it to sign. The sections that follow walk through how that foundation gets built out: first into a verified boot sequence, then into encrypted memory, then into a protected path across the PCIe bus, and finally into an attestation report a workload owner can check before trusting the machine with anything sensitive.

The on-die root of trust and secure boot chain before workload execution

Confidential computing on the H100 starts before any workload touches the GPU. When an H100 boots in CC-On mode, a hardware root of trust built into the die anchors a measured boot sequence that has to finish before the GPU will accept a single instruction. Only signed, attested components are allowed to execute in CC mode, and unsigned or tampered firmware stops the boot process outright.

That boot sequence produces something concrete: cryptographic measurements, hashes of each firmware component as it loads, that later form the raw material for the attestation report covered further on. Every H100 ships with its own unique elliptic curve cryptography keypair and a matching public certificate, provisioned at manufacture. NVIDIA backs that identity with an OCSP service, so anyone relying on a given GPU can check that its certificate is valid and that the device has not been revoked before trusting it with anything.

All of this defines what security researchers call the firmware trusted computing base, or TCB: a deliberately narrow set of signed, measured components. Anything outside it is untrusted by construction, not by configuration choice. That distinction carries the entire chain forward. The boot sequence does not ask the GPU to behave correctly; it checks, cryptographically, that the GPU has in fact done so, and it refuses to proceed otherwise.

VRAM partitioning and encryption that keeps active computation hidden from the host

With a trustworthy boot behind it, the GPU turns to encrypting memory so it no longer sits in plaintext while the GPU works on it. In CC-On mode, the H100 splits its device memory into a protected Compute Protected Region, or CPR, and an unprotected region outside it. NVIDIA's architecture documentation states the CPR can cover up to 90% of GPU device memory, and hardware firewalls block any outside access to it. The PCIe firewall blocks the CPU from reaching most GPU registers and all CPR memory, while the NVLink firewall blocks other GPUs on the same system from reaching into a peer's CPR.

Data that does need to leave the CPR does not leave in the clear. A dedicated Confidential Computing Engine built into the GPU die encrypts data headed outside the CPR using AES-256-GCM before it departs, via the GPU's DMA engine, which handles writes made outside the protected region rather than every write that touches HBM. The host software stack, including the hypervisor, never has access to it. That is a hardware constraint built into the chip that a misconfigured host could not relax.

The practical consequence follows directly: a hypervisor with full DMA access to the GPU's entire physical address space reads nothing but ciphertext. This is the direct closure of the blind spot named at the start of this piece. VRAM is no longer exposed as plaintext to the layer of software that used to have unrestricted visibility into it. And the protection does not come at the cost of raw performance: GPU compute throughput and HBM memory bandwidth run at parity with non-CC mode, because the Confidential Computing Engine does not slow down internal computation. Whatever overhead confidential computing introduces does not occur inside the GPU's own compute path, a point that becomes central later when the discussion turns to where performance overhead concentrates instead.

Encrypted bounce buffers and PCIe firewalls protecting the CPU-GPU boundary

Encrypting VRAM solves one half of the problem. Data must also travel safely into and out of that protected memory, since the PCIe bus connecting CPU and GPU was identified earlier as an open interception surface. Data moving between the CPU trusted execution environment and GPU CPR memory travels through an encrypted bounce buffer in shared system memory, keeping the PCIe bus opaque to the host.

The mechanics work like this. The NVIDIA driver, running inside the CPU's own trusted execution environment, encrypts data before writing it into the bounce buffer, and only the GPU's DMA hardware engine decrypts that data as it lands in the CPR. Command buffers and CUDA kernels, the actual instructions telling the GPU what to compute, cross the PCIe bus encrypted and signed as well, closing off a path an attacker could otherwise use to tamper with execution itself rather than just read data.

Supporting that encrypted channel is a fairly deep key structure. Eight Link Compression Engines handle secure host-to-device and device-to-device communication, each carrying distinct keys for both directions and for both user and kernel mode, which works out to 32 derived keys in total. That is a deliberately granular key hierarchy, built so that no single compromised key exposes more of the channel than strictly necessary.

None of this operates in isolation from the CPU side. The H100 is designed to work alongside CPU trusted execution environments, specifically Intel TDX, AMD SEV-SNP, or Arm CCA, which encrypt the CPU-side memory feeding into the bounce buffer in the first place. PCIe bus traffic between CPU and GPU also travels in the clear on non-CC hardware, opening an interception surface for anyone with physical or privileged access.

Attestation and verifying the GPU's security state before trusting it with sensitive data

Every protection described so far, secure boot, the CPR, the encrypted bounce buffer, exists inside the machine. Once secure boot finishes, the H100 produces a cryptographically signed attestation report, and a workload owner can inspect that report before releasing model weights or sensitive data to the GPU.

That report is a signed set of the exact measurements taken during boot, and it ties the GPU's own identity, established through its per-device ECC keypair, to the specific firmware state measured at startup. Delivery of that report runs over an SPDM session, which connects the GPU securely to the driver inside the CPU TEE and forms the channel the attestation report travels through. Underneath the session, authentication runs on standard public key infrastructure: the GPU's certificate chains back to NVIDIA's certificate authority, the OCSP service lets a verifier check whether a specific device has been revoked, and NVIDIA's attestation service stores each nonce alongside the device's certificate hash for 24 hours to block replay attacks.

What makes this practical rather than merely secure is how little it demands of developers. Running a CUDA application in CC-On mode requires no code changes at all, provided the hardware, drivers, and attestation report are all in order; the workload simply proceeds if attestation passes and gets blocked if it does not. A security model that required rewriting application code would face a much steeper climb into production use than one that asks only for a passing attestation check.

Taken as a whole, attestation changes what a workload owner has to trust. Instead of trusting a cloud provider's internal operational controls, a password policy, an access log, an auditor's sign-off, the owner trusts hardware cryptography and checks it directly through attestation. That shift matters most for exactly the workloads where it is hardest to extend blind trust to a third party: inference over regulated health records, financial data subject to strict access rules, or any model whose weights represent protected intellectual property.

The three operating modes and their security-versus-observability trade-offs

None of the protections above are forced on every H100 deployment unconditionally. Confidential computing operates in one of three persistent modes, CC-Off, CC-On, and CC-DevTools, each with a distinct security-versus-observability trade-off baked in by design.

CC-Off is the GPU operating as it always has: no CC-specific protections engaged, full profiling and debugging tools available. Turning it off is a deliberate design choice. CC-DevTools exists to answer the resulting gap in visibility: it re-enables profiling and debugging by turning security protections back off, which makes it suitable for development and benchmarking but not for anything running in production.

Switching between these modes is not something a running instance can do on its own. The CC control bit lives in the GPU's EEPROM and gets set using gpu_cc_tool.py in-band or through an out-of-band API, which makes mode selection persistent rather than a runtime toggle. Turning on CC-On for an instance is a configuration decision made at the host level before a tenant ever takes possession of the machine, not a setting a tenant can flip after the fact. A provider planning to offer confidential GPU instances has to decide that commitment in advance, as part of how the fleet gets provisioned, not as an afterthought layered on top of existing capacity.

Performance overhead and workload size

Every security feature carries some cost, and the honest question about H100 confidential computing is where that cost concentrates and how much it matters in practice. The overhead concentrates almost entirely in CPU-GPU data transfer across PCIe, and it shrinks toward negligible as the model being served gets bigger and sequences get longer.

A benchmark study on H100 CC performance found time-to-first-token overhead around 19% for a smaller Llama-3.1 model running with TEE protection on, but essentially no overhead at all for the larger model in the same family, because the larger model's own computation time dwarfs whatever the encrypted transfer costs. What limits throughput in the cases where overhead is visible is the CPU's own encryption performance, which caps the CPU-GPU interconnect at roughly 4 GBytes per second, and that cap is the binding constraint specifically for small-model, short-sequence workloads.

Inside the GPU itself, nothing changes. Raw compute throughput and HBM memory bandwidth are untouched by CC mode, since the entire overhead lives in the I/O path rather than in computation. Practical implication: the workloads most in need of confidential computing, large-model inference on sensitive data, are also the workloads least harmed by CC mode's overhead. That is not a coincidence baked in by NVIDIA's design intent so much as a consequence of where the bottleneck physically sits, but it works out favorably for exactly the deployments where privacy guarantees matter most.

The three genuine limitations of H100 CC mode that operators must plan around

None of this is a complete or unconditional solution, and an honest account of the architecture has to say where it falls short. The most significant limitation is the multi-GPU gap. GPU-CC, as released on Hopper, does not support multi-GPU configurations, and the NVLink connections between GPUs are not encrypted when operators fall back to the workaround mode available for multi-GPU systems. That workaround is Protected PCIe, or PPCIE, mode, built for HGX H100 and H200 8-GPU systems specifically to give operators a usable path toward multi-GPU deployments despite the gap.

Large model training and large-scale inference increasingly depend on exactly this kind of multi-GPU scaling, so the gap concentrated in multi-GPU configurations lands squarely on some of the highest-value workloads confidential computing was built to protect.

That gap also explains why discussion of this architecture tends to point forward rather than stopping at Hopper, and why tools built with privacy as a hard constraint, rather than a configurable option, such as Confidant AI, a privacy-preserving AI assistant whose architecture is built from the ground up to keep user data out of reach rather than retrofitted for it, are designed from the ground up rather than patched to meet that bar. The limitations named here, chiefly the absence of native multi-GPU support and the unencrypted NVLink path under PPCIE, mark the specific territory later GPU architectures have reason to address. Understanding exactly where the H100 draws its boundary matters for any operator deciding how to deploy it today, and it matters just as much for understanding what comes next.