Bring Your Own Key Is Not Enough for True LLM Output Confidentiality

Bring Your Own Key is not enough for true LLM output confidentiality because it controls encryption or account credentials and leaves access during computation unprotected. A zero-trust AI architecture for enterprise must protect prompts and outputs at inference time, when ordinary serving systems expose them in plaintext. Preventing an AI vendor from reading company queries requires an architectural boundary.

Why "Bring Your Own Key" has become the default privacy answer for enterprise LLM deployments

Enterprise procurement and compliance teams need controls they can check before they approve an LLM deployment. BYOK gives them identifiable responsibilities, account relationships, and audit evidence. It has therefore become a standard security checkpoint, particularly where regulated data requires clear governance. Its appeal rests on moving important decisions from the provider’s defaults into the enterprise’s hands.

Two distinct controls travel under that label. With API-key BYOK, an enterprise supplies its own model-provider credential for OpenAI, Anthropic, or AWS Bedrock to a third-party platform. Calls are billed and logged under the enterprise’s provider account, and its negotiated data processing agreements govern the data. This preserves a direct provider relationship even when another platform supplies the interface through which employees work.

Encryption-key BYOK, also called Customer-Managed Keys or CMK, gives the enterprise authority over cryptographic material. The enterprise generates a key and imports it into a service such as AWS KMS, Azure Key Vault, or Google Cloud KMS to encrypt stored data. The customer controls the key’s lifecycle while the provider handles key storage and cryptographic operations. This division gives the enterprise more authority without requiring it to operate every part of the encryption infrastructure.

Both arrangements answer legitimate governance needs. Direct billing helps with cost attribution, account-level logging supports cleaner audit trails, and key-lifecycle authority gives security teams a defined control to administer. Regulated sectors are right to require these capabilities as a floor. Their popularity is understandable because they make responsibilities that might otherwise remain implicit more explicit.

What encryption-key BYOK actually controls, and where that control ends

Encryption-key BYOK governs encryption around data at rest and in transit while leaving the live computation that produces an LLM response outside its control. “Bring your own” describes the key’s origin and the customer’s lifecycle authority. The key may still enter provider infrastructure while it is in use.

In the common imported-BYOK model, key material is inside the provider’s hardware security module while it carries out cryptographic operations. The HSM is designed to prevent that material from being accessible in plaintext outside it. That is a meaningful protection, but it differs from keeping the key entirely outside the provider’s environment. A provider system can carry out an authorized cryptographic operation without letting a person see the raw key.

The distinction matters because protecting the key and excluding access to decrypted data are separate goals. Once encrypted material is decrypted for ordinary LLM inference, customer governance over its storage key does not shield the resulting plaintext. Confidant AI treats this as an inference-architecture problem, with a design intended to make user data inaccessible to the operator. That addresses a different boundary from the one encryption-key BYOK was built to protect.

HYOK, or Hold Your Own Key, and External Key Store models change where keys are kept by holding them in infrastructure that the customer controls. HYOK keeps keys out of provider memory, though key operations across a network can add latency that teams must check for each workload. It should not be confused with imported BYOK merely because both give customers key authority. Keeping the key separate also does not, on its own, keep data confidential while the model computes.

Legal compulsion exposes the importance of these distinctions. Under the CLOUD Act, US law enforcement can compel providers to decrypt customer data stored anywhere in the world, often without notifying the customer. In the imported-BYOK model, the provider’s systems hold key material and use it for encryption and decryption. Even a perfectly implemented arrangement therefore leaves the provider with technical access to key material through its cryptographic systems when legal processes compel cooperation.

What API-key BYOK genuinely changes about privacy, and what it does not touch

API-key BYOK changes the account and contract under which an inference request runs. That can substantially improve data handling because provider API terms can be more protective than consumer terms. OpenAI uses API data to train or improve models only if the customer explicitly opts in, while its consumer product uses that data by default. Anthropic does not keep API conversation content by default and uses any data it keeps for model training only with express permission.

The account’s terms matter as much as possession of the credential. Google Gemini routes prompts from a billed project away from the free-tier path, where usage develops Google’s products and human reviewers can access data. Supplying a free-tier key does not secure the protections of the billed path. The privacy benefit comes from the service arrangement that the key selects.

This is a real improvement, but it leaves the computation unchanged. Prompts and outputs still exist in plaintext on ordinary provider infrastructure during inference, regardless of whose credential initiated the call. The gap explains the relevance of privacy-preserving designs such as Confidant AI’s, which make inference confidentiality a core property.

The request path can also include more infrastructure than the credential suggests. Some AI coding assistants build prompts on their own servers, so requests pass through the vendor’s infrastructure even when users supply their own API keys. A study of AI chatbot apps on iOS also found that network traffic often exposed paid AI access, with some keys sent in plaintext. Only a minority had fixed the problems months after disclosure.

The actual vulnerability — what must happen to prompts and outputs during inference

During ordinary LLM inference, prompts and outputs must exist in plaintext on the serving infrastructure. The model must read the prompt to generate a response. Encrypting that prompt before processing, or encrypting the response afterward, does not protect the interval in between.

In a standard cloud-hosted LLM, the service provider controls the serving hardware and software and can read inference data in plaintext. Storage encryption does not block that access because the data is already decrypted for use. A promise not to inspect prompts limits what a provider is allowed to do, while a compute-time boundary limits what it can do.

The exposure also extends beyond the most obvious prompt and response buffers. LLM serving engines accelerate inference by sharing key-value attention caches across requests. The KV-cache is often transmitted and stored in plaintext, creating an attack surface within the inference layer. “Shadow in the Cache” at NDSS 2026 and concurrent research documented this surface, showing exploitable behavior.

This distinction is especially consequential when prompts contain PII, PHI, or financial information. JPMorgan, Citi, and Goldman Sachs restricted ChatGPT use, while healthcare providers raised HIPAA compliance concerns. The underlying concern is not adequately described as a failure to manage encryption keys. With standard inference architecture, sensitive information is exposed while the model does the work the enterprise wants it to do.

Agentic AI broadens that exposure because autonomous agents can access substantial amounts of sensitive data and send it to cloud infrastructure unpredictably. The confidentiality boundary must therefore cover computation as well as a known document upload or a single prompt written by a user. This changes the frame of the enterprise privacy discussion. The central property is not who brought the key, but who can read the data while the model works.

What closing the inference gap actually requires — architectural approaches and their trade-offs

Genuine output confidentiality requires an inference environment in which the operator cannot read plaintext. Confidential computing uses Trusted Execution Environments, or TEEs, to set that boundary with enclaves whose protection comes from hardware. The cloud operator cannot access the enclave even with root privilege on the host. Whether the user can independently verify the enclave’s attestation still needs to be established.

Apple put this into practice on June 8, 2026, when it extended PCC beyond its own data centers to Google Cloud. The deployment combines three independent hardware trust layers, namely Nvidia Confidential Computing on Blackwell GPUs, Intel TDX on CPUs, and Google’s Titan chip. Its attested enclave spans both processor and accelerator. This is the first multi-vendor, CPU+GPU confidential inference deployment at global consumer scale, making compute-time protection more than a research proposition.

Performance evidence also makes the trade-off more specific. Confidential computing on NVIDIA Hopper GPUs shows minimal computational overhead within the GPU, while data transfer accounts for the main overall penalty. That weakens the blanket objection that confidential GPU inference is inherently too slow. It does not remove the need to assess the complete workload, because GPU computation and movement of data contribute differently to total performance.

Talaria, a 2026 academic framework, explores a different partition of the LLM pipeline. Sensitive, weight-independent operations run inside a Confidential Virtual Machine that the client controls, while weight-dependent computations run on cloud GPUs. Its Reversible Masked Outsourcing protocol, or ReMO, uses hybrid masking to reversibly obscure intermediate data before outsourcing. The design keeps client data safe without revealing the cloud’s model weights, so the confidentiality boundary covers selected parts of the computation.

SafeKV, published in 2025, integrates detection and isolation into the serving runtime to eliminate cross-tenant reuse of sensitive KV-cache blocks. On large LLM backends, it substantially reduces time-to-first-token overhead compared with full isolation and raises throughput. Its contribution is narrower than complete inference confidentiality, but it shows that a specific compute-time attack surface can be addressed without prohibitive performance cost.

These approaches expose less data, but they still require trust. TEEs shift trust toward the hardware manufacturer and enclave software, rather than providing a mathematical proof of confidentiality. Fully homomorphic encryption would remove the plaintext-at-computation requirement entirely, but its computational overhead remains prohibitive for practical LLM serving at current scale. The choices ahead are therefore between architectures that differ in what they protect and what they cost, and go beyond a single replacement key-management feature.

How do you prevent an AI vendor from seeing company prompts and data

Start the evaluation with the whole request lifecycle, including prompt construction rather than only the final API call. Ask where plaintext is held or processed and which operator can access it at each point. A system may decrypt data inside a protected environment without making it readable to the host operator. The useful distinction is therefore between plaintext that exists for computation and plaintext that the vendor can reach.

For an enclave-based service, ask whether inference runs inside a hardware-attested enclave and whether users can independently verify that attestation. Ask whether the provider publishes a verifiable log of production node images so users can compare the running software with the audited codebase. In a multi-tenant deployment, ask how the KV-cache is isolated and whether that isolation has been tested against timing-based fingerprinting attacks. These tests of the execution boundary complement a key-management questionnaire.

Legal exposure belongs in the same assessment. Ask what the provider can disclose under compulsion and whether its architecture limits that capability. Confidant AI is built from the ground up around the principle that privacy must be architectural. Its inference architecture is designed so that user data is never accessible to Confidant as operator, rather than protected only by a policy that legal process or a configuration change could override.

That design principle makes Confidant AI an option for enterprises that work with PII, PHI, financial information, or proprietary business context. It addresses the specific shortcoming of BYOK by making inference confidentiality the core property. The principle is not, by itself, evidence of any particular enclave implementation, attestation mechanism, or cache-isolation method. Those details still need to be checked with the same questions about architecture that apply to other approaches.

API-key BYOK still has a place in this stack because it provides audit trails, cost attribution, and contractual terms through a privacy-preserving gateway. Meanwhile, CMMC Level 3 and FedRAMP High are pushing toward models in which providers demonstrably cannot access customer data. Italy’s temporary ChatGPT ban and later fine over training on personal data also show pressure from regulators, though the Court of Rome annulled the fine in March 2026 on jurisdictional grounds. The enterprise decision is not whether keys matter, but whether the architecture protects the moment that keys cannot.