CPU-GPU Confidential Channel Establishment with AMD SEV-SNP

Confidential computing solves a problem that at-rest and in-transit encryption never touched: what happens to data while it's actually being processed. AMD SEV-SNP handles that problem for CPU workloads, encrypting guest memory and register state with keys the hypervisor never sees. Once a GPU enters the picture, that guarantee doesn't automatically extend across the PCIe bus, and stitching it back together is one of the more intricate engineering problems in modern infrastructure. Most descriptions of "confidential AI" skip past this part entirely, and it's the part that actually costs something.

This piece walks through how that stitching happens: two separate hardware roots of trust, the AMD Secure Processor on one side and the GPU's own on-die security engine on the other, agreeing to trust each other across a bus that neither one fully controls. It is not a config file or a driver flag. It is a layered handshake involving memory classification, cryptographic attestation, and DMA access rules, and each layer carries a real, measurable performance cost, one the industry has been strangely quiet about until recently.

What AMD SEV-SNP actually controls before the GPU enters the picture

SEV-SNP runs on two pieces of hardware doing distinct jobs. The AMD Secure Processor manages keys and policy, and a dedicated encryption engine sitting in the memory controller encrypts and decrypts on every single memory access. Neither of these is a software layer bolted on after the fact. They are built into the silicon, and the hypervisor has no path around them. That is precisely the point: software-level trust is exactly what this architecture refuses to rely on.

The mechanism that makes SEV-SNP work is a data structure called the Reverse Map Table, or RMP. It is system-wide, and it tracks the security classification of every physical page of memory on the machine. A page is either guest-private, meaning it is encrypted and owned by the guest with the hypervisor locked out of writing to it, or it is shared, meaning the hypervisor can see and touch it. Try to access a private page from the wrong privilege level, and the hardware catches it at the point of access, not after the fact, not through an audit log. It is enforced in silicon, so the integrity guarantee holds even if every layer of software above it, hypervisor included, turns hostile.

The guest keeps its sensitive code and data in private memory and uses shared memory only when it deliberately wants to talk to the host. That communication runs through a narrow, defined channel called the GHCB, the Guest Hypervisor Communication Block, which lets the guest ask the hypervisor for services (interrupts, I/O, that sort of thing) without ever trusting it. VM Privilege Levels and an encrypted VM save area round this out, letting the guest cooperate with a hypervisor it fundamentally does not trust. Cooperation without exposure, essentially.

None of this matters to a remote party unless it can be checked from the outside, so SEV-SNP also produces a hardware-signed measurement of the guest's initial state, tied back to AMD's certificate chain. A verifier somewhere else on the network can check that measurement and confirm exactly what code is running before it releases any secrets to the machine. The overhead for all of this, on the CPU side alone, is small: roughly 4% on integer throughput benchmarks and around 2% on Java workloads. Keep that 4% in your back pocket. Whatever tax shows up once a GPU joins the picture is coming from somewhere else entirely, and it is a much bigger number.

How the GPU establishes its own root of trust before the channel can open

NVIDIA's Hopper generation, H100 and H200, was the first GPU line to support a confidential computing mode, introduced in 2023. Getting there means the GPU has to do its own version of what SEV-SNP does for the CPU, and it has to do it before it ever talks to the host. That sequencing is not incidental. It is the design.

Research published as arXiv:2507.02770 points to three on-die security engines doing the work: the Foundation Security Processor (FSP), the GPU System Processor (GSP), and a Secure Processor called SEC2. NVIDIA hasn't published full specs on how these three interact, and the available picture relies on patent filings and instrumented testing rather than an official architecture document. Worth flagging as a limit on how precise this picture can get.

When the GPU boots into what NVIDIA calls CC-On mode, a hardware root of trust baked into the die anchors the whole chain. Firmware and microcode go through a secure, measured boot, verified and recorded before any of it executes. A hardware firewall gets configured to block unauthorized access to protected memory regions before PCIe even comes up. The order matters here: the GPU locks itself down internally first, and only then does it open the bus. Flip that order and the whole model falls apart, since there would be a window where an unverified device sits exposed on an active bus.

Device memory on the GPU gets split into two logical regions. The Compute Protected Region, or CPR, is the slice of HBM (high-bandwidth memory) that runs at full speed inside the trust boundary. The unprotected region handles communication with the host and hypervisor. In CC mode, the GPU's firmware itself becomes part of the Trusted Computing Base, meaning a guest has to check those firmware measurements before it can trust the device at all.

So the handshake covered in the next section is not establishing trust from scratch. It is authenticating a device that already locked its own doors before anyone showed up.

The SPDM handshake: how the CPU TEE and GPU authenticate each other across PCIe

Once the GPU has established its own trust state, the CPU TEE and the GPU need to find each other and agree on a shared secret, and they do it using SPDM, the Security Protocol and Data Model, an industry standard built for device authentication and key exchange over transport-agnostic channels.

The driver inside the confidential VM starts this process, not the hypervisor. That is a deliberate choice: the hypervisor is the untrusted party in this whole arrangement, and letting it kick off the trust negotiation would put the fox in charge of scheduling the henhouse audit.

The sequence runs in stages. First, the driver sends GET_CAPABILITIES and NEGOTIATE_ALGORITHMS messages to figure out which cryptographic algorithms both sides can actually use. The GPU answers with its certificate chain, rooted in its on-die root of trust, and the driver checks that chain against known-good roots. Next, the driver asks for a full attestation report from the GPU: signed measurements covering firmware state and microcode version. Only if those measurements match what is expected does the process continue. Then comes SPDM's KEY_EXCHANGE step, a Diffie-Hellman-style exchange that produces session keys known only to the CVM driver and the GPU's security processor. From that point on, everything moving across that session is encrypted and integrity-checked.

The trust flows both ways here, which is easy to miss. The GPU is not just proving itself to the CPU. It also gets confirmation that it is talking to a genuinely attested CPU enclave, not a hypervisor pretending to be one.

There's an operational wrinkle worth flagging, the kind of detail that only shows up once someone runs this in production. If the shared session keys get destroyed, the session ends, and the GPU will not let SPDM re-establish without a Function Level Reset, which fully wipes and resets the GPU. That is a heavier operation than it sounds, which is why something like the nvidia-persistenced daemon has to keep running continuously just to hold the session open. Losing that session is not a quick reconnect. It is a full device reset. On Hopper, the PCIe link itself does not protect these messages either. All the cryptographic protection here comes from the SPDM layer sitting on top of an otherwise unprotected bus, a gap that SEV-TIO exists specifically to close.

Memory page classification across the two domains: shared pages, bounce buffers, and why DMA is the bottleneck

Once the SPDM session is live, actual data movement still has to respect the memory rules on both sides at once. This is where the architecture starts costing something measurable, and where the real argument of this piece sits.

SEV-SNP's private pages, the ones marked C=1 in the guest's page tables, cannot be DMA targets for any device the hypervisor controls. The RMP enforces that rule at the hardware level, full stop. On Hopper, the GPU is not bound into the guest's trust domain at the hardware level for DMA purposes. That leaves exactly one option: any data headed to the GPU has to pass through shared, C=0, memory, the kind the hypervisor can see.

So the CVM has to do the work itself. It encrypts data before dropping it into a shared page, and decrypts data coming back out of one. The staging area for this is called a bounce buffer, sitting in shared memory: copy from private memory, encrypt it, place it in the shared buffer, and only then does the GPU's DMA engine pick it up. The return trip runs the same steps in reverse. The GPU writes into shared memory, the CVM decrypts it, then copies it into private memory.

That copy-and-encrypt cycle, running on the CPU, becomes the binding constraint for I/O-heavy workloads. It does not matter how much HBM bandwidth the GPU has sitting idle if every byte first has to crawl through a CPU-side encryption step, and it's worth being blunt here: on Hopper, there is no way around it. The bounce buffer is not a shortcut some vendor took. It is a correct, faithful implementation of the threat model SEV-SNP defines, and it is also a tax, paid on every single transfer, that Hopper's architecture has no mechanism to waive.

That tax lands unevenly depending on the workload. Compute-heavy jobs spread the transfer cost across more GPU work, reducing its relative impact. Latency-sensitive workloads feel the overhead more directly, as the benchmark data on time-to-first-token confirms. That split is exactly what the numbers in the next section show.

What the overhead actually looks like in practice on H100 and H200 systems

Numbers help make this concrete. Research published as arXiv:2409.03992 in 2024 found average overhead across LLM inference tasks on H100 systems under 9%. That is the headline figure, but the breakdown by model is where the actual argument lives, and it's the part most summaries leave out.

Running vLLM v0.5.4 in confidential computing mode, Meta-Llama-3.1-8B-Instruct showed a 6.85% overhead in tokens-per-second and 3.22% in queries-per-second. Phi3-14B-128k came in lower, at 4.58% TPS overhead and 2.31% QPS overhead. Llama-3.1-70B, the largest model tested, actually showed a TPS overhead of -0.13%, effectively zero. That pattern is not noise. It confirms the bounce buffer mechanism directly: bigger models keep the GPU busy for longer stretches per request, so the fixed cost of shuffling data through the bounce buffer gets amortized across more compute and shrinks toward irrelevance. Anyone picking a model size for a confidential deployment on accuracy alone, without weighing this, is leaving throughput on the table for no reason.

A separate set of fixed-rate experiments on H100 under Intel TDX, published as arXiv:2607.19353, tells a sharper story about latency specifically. Confidential mode increased average time-to-first-token by 21.8% for Mistral-7B and by 27.8% for Qwen3-30B-A3B, while global token throughput dropped 17.7% and 21.1% respectively. Time-to-first-token is a pure latency metric, and it is far more sensitive to bounce-buffer overhead than raw throughput is. That gap between the throughput numbers and the TTFT numbers is really the whole architecture in miniature: small, latency-sensitive transfers pay far more than bulk, throughput-oriented ones. Anything built around fast interactive response, a chatbot, a real-time agent, should expect to feel this a lot more than a batch summarization job ever would.

Put together, for a typical LLM query the overhead stays under 5%, and longer sequences on bigger models eat relatively less of a penalty because GPU compute time swamps the transfer cost. These figures are not just descriptive. They are the baseline the next architecture, SEV-TIO, is explicitly built to erase.

The composite attestation report: binding CPU and GPU measurements into a single verifiable claim

Attestation is not only an internal handshake between the driver and the GPU. It has to produce something a party outside the machine can actually check, which is what turns "trust me" into "verify this."

The mechanism is a composite attestation report: the GPU's hardware attestation, covering its own firmware and configuration, gets combined with the AMD SEV-SNP report from the CPU TEE into one signed bundle of evidence. On the verification side, NVIDIA runs a service called the NVIDIA Remote Attestation Service (NRAS), which checks GPU evidence against known-good reference values pulled from the Reference Integrity Manifest Service, essentially a registry of approved firmware, VBIOS, microcode, and hardware configurations.

That GPU attestation report carries cryptographic hashes of the device's firmware and configuration, covering the state of the GPU at the time of attestation. Only once the CVM checks out as verified and unmodified does anything sensitive, like a model's decryption keys, actually get released into it. Attestation is the gate here, not a formality bolted on beside it.

Two practical details are worth knowing. A single attestation request can cover multiple NVIDIA confidential-computing GPUs at once, which matters a great deal for anyone running multi-GPU inference nodes rather than single-card setups. And attestation itself is a one-time cost at startup, on the order of seconds per provisioning event, not something paid on every inference call, so it does not stack onto the per-transfer overhead discussed above.

The composite report is where the two separate trust anchors, the AMD Secure Processor and the GPU's own root of trust, get formally joined into a single claim a verifier can check. The whole channel is only as trustworthy as that report turns out to be, which is why vulnerabilities in the attestation chain matter so much. Case in point: CVE-2024-56161, rated 7.2 (High) and reported to AMD by Google researchers, involved improper signature verification in the AMD CPU ROM microcode patch loader. An attacker with local administrator privilege could exploit it to load malicious CPU microcode, undermining confidentiality and integrity of a confidential guest running under SEV-SNP. AMD shipped microcode and SEV firmware updates across affected EPYC generations, from Naples through Genoa (Zen 1 through Zen 4), and later added Turin (Zen 5) to the advisory in April 2025. What's notable is how the fix gets confirmed: a confidential guest can check whether the mitigation is actually in place by reading its own SEV-SNP attestation report. Attestation is the foundation trust gets built on here, and it doubles as the mechanism for verifying a patch actually landed.

SEV-TIO: how eliminating the bounce buffer requires changing the trust architecture, not just the software

AMD describes SEV-TIO, Trusted I/O, as the next major evolution of SEV, and the goal is direct: extend the same trust boundary SEV-SNP builds around CPU memory out to PCIe devices themselves.

It is built on a stack of existing industry standards rather than a from-scratch protocol. TDISP, the TEE Device Interface Security Protocol defined by PCI-SIG, spells out how a device's virtual function gets bound into a guest's trust domain. IDE, Integrity and Data Encryption, adds link-layer encryption and integrity protection directly on the PCIe bus. SPDM carries the authentication and attestation work, the same protocol already doing that job on Hopper, except now it sits on top of a PCIe link that is actually protected rather than a bare one.

The structural change is what matters here, and it's worth being direct about what most descriptions of SEV-TIO get wrong: this is not a software patch that makes the existing bounce buffer faster. It changes what the hardware is allowed to do. The device authenticates to the guest over SPDM, but now the PCIe link itself is shielded by IDE, so the bus stops being an unprotected transport the way it is today. The guest checks the device's identity and firmware through attestation, and once that checks out, the device gets bound into the guest's trust domain at the hardware level. That last step flips the RMP rule from earlier: once the IOMMU sees the device as properly attested and trusted, it permits that device to DMA straight into guest-private, C=1, memory. The classification rule hasn't changed at all. What changed is that the device now qualifies to touch private memory directly, something no amount of clever driver code on Hopper could ever grant it.

The consequence is that the entire bounce-buffer dance, copy, encrypt, stage, DMA, decrypt, copy, goes away. Data moves directly between GPU HBM and guest-private DRAM, with confidentiality and integrity enforced by hardware on the wire itself rather than by CPU cycles spent re-encrypting everything in software. Whether TIO is actually active shows up in the attestation report, so a remote verifier can tell whether a given session used the direct path or fell back to bounce buffering. Both paths remain valid under their own threat models, worth saying plainly since one is not a security downgrade from the other. One of them just stops paying the tax detailed a few sections back, and for latency-sensitive inference, that's the whole ballgame.

Where SEV-TIO support stands today and what the Blackwell generation adds

Support for this is still catching up to the standard, and that gap is the honest answer to the question everyone actually wants answered: can you run this today? Mostly not yet. Current-generation platforms, 4th Generation AMD EPYC processors alongside a fifth-generation part from another CPU vendor, represent where the industry sits on TDISP and IDE readiness, and broader deployment depends on both CPU platforms and GPU generations lining up on the same set of standards at the same time.

That alignment hasn't fully happened, and it's worth resisting the temptation to treat SEV-TIO as already shipped just because the specification is public and the underlying standards (TDISP, IDE, SPDM) are mature individually. A spec being finished is not the same thing as silicon shipping with it turned on, and the gap between those two facts is exactly where the Blackwell generation of GPUs matters. That is the point where the direct-DMA path described above stops being a specification sitting in a PCI-SIG document and starts being something real workloads actually run on, with the bounce buffer finally retired rather than merely explained away.