Privileged Document Handling Requires Inference Architecture Not Just Privacy Policies

AI tools for privacy-conscious professionals need more than a promise not to train on client data. For privileged documents, the key questions are who directs the work, who can access information during inference, and what governance supports confidentiality. Privacy policies and enterprise contracts help, but they cannot substitute for an architecture that limits access when sensitive information is processed.

Why privilege questions are being decided at the moment of transmission, not in court

Vendor privacy policies and data processing agreements set out what vendors must do, but courts reviewing AI-assisted legal work also look at how the information was shared. Was the platform being used as part of a professional relationship? Could the user reasonably expect confidentiality under its terms? Those questions cover what happens when the user shares information as well as what the vendor later does with a document. A contractual promise is not an architectural guarantee.

That distinction changes when the practical decision occurs. A court may determine protection later, but the facts supporting or defeating it begin to take shape when the user transmits information. Attorney involvement, platform access and permitted disclosures cannot simply be rewritten after a discovery request arrives. Treating promises governing data use as proof that confidential processing was possible is a category error broader than careless use of a consumer tool.

What the Heppner rulings actually held, and why the reasoning goes beyond one bad actor

United States v. Heppner made that distinction concrete. In S.D.N.Y., Judge Rakoff issued a bench ruling on February 10, 2026, and a written opinion on February 17, 2026, denying attorney-client privilege and work-product protection for documents generated using Claude. After receiving a grand jury subpoena, Heppner had entered questions about the government’s investigation into Anthropic’s consumer-tier platform, including facts learned from counsel. Federal agents seized devices that held roughly thirty-one documents made up of his prompts and Claude’s outputs.

The court identified two independent defects in the privilege claim. Claude was not an attorney, owed no fiduciary duty and was not functioning as counsel’s agent. Separately, the court examined Anthropic’s privacy policy, which permitted collection of prompts and outputs, model training and disclosure to third parties, including government authorities without a subpoena. Those terms defeated a reasonable expectation of confidentiality. The fact that Heppner wanted legal help did not fix either defect.

Work-product protection failed on a different ground. Heppner had prepared the documents on his own initiative, rather than at counsel’s direction, so the court did not regard them as materials “prepared by or at the behest of counsel in anticipation of litigation.” Yet the court left an important opening on attorney-client privilege. Had counsel directed the use, Claude might arguably have functioned as counsel’s agent. Sending the documents to counsel afterward could not supply that relationship retroactively.

How UK courts reached the same structural conclusion through different doctrine

Hamid 2026 UKUT 81 brought a related distinction into English legal professional privilege. The UK Upper Tribunal (Immigration and Asylum Chamber) directly addressed confidential material uploaded to what it described as open-source AI tools. Its comparison was between publicly available tools such as ChatGPT and closed systems operating within a secure network.

The Tribunal observed, by way of obiter dictum, that uploading privileged material to an open-source platform breaches confidentiality and waives privilege. That qualification is important. The observation should not be presented as a holding resolving every form of enterprise AI deployment. Nevertheless, its reasoning reaches beyond immigration work because it asks whether the environment preserves confidentiality. Read with Heppner, it points toward scrutiny of the actual disclosure arrangement rather than reliance on the user’s intention to keep a document private.

Why Warner v. Gilbarco and earlier TAR cases show that AI use can preserve protection (and what made the difference)

Warner v. Gilbarco shows why that scrutiny does not yield a universal rule against AI. In E.D. Mich. on February 10, 2026, Magistrate Judge Patti denied a request to compel a pro se plaintiff’s ChatGPT sessions. The court treated the platform as a tool and found that its use preserved protection against waiver through disclosure to an adversary. It also found that the materials were prepared in anticipation of litigation and that production would be disproportionate to the case’s discovery needs. Attorney direction was not independently required for work-product protection.

The distinction between doctrines is essential here. Warner did not establish attorney-client privilege for a conversation without an attorney, nor did it establish hardware isolation as a condition of work product. Its reasoning instead shows that using an outside AI platform need not itself waive protection. That limits how far Heppner can be generalized. The cases require attention to the protection claimed and the circumstances of creation and disclosure.

Earlier technology-assisted review decisions provide narrower support for this approach. Da Silva Moore v. Publicis Groupe approved predictive coding for appropriate eDiscovery work on grounds of proportionality and reasonableness under the Federal Rules of Civil Procedure. Brown v. BCA Trading Ltd extended that logic to contested TAR applications, grounding approval in proportionality and the Pyrrho factors. Neither decision established an inference-security test. Together, these cases suggest three things to assess, attorney direction, how the tool fits into professional work and the platform’s confidentiality architecture, though these do not form a legal test that applies to every case.

What ABA Formal Opinion 512 adds — the inference-layer risk that no policy can cure

ABA Formal Opinion 512, issued July 29, 2024, shifts the focus from what courts have decided to what lawyers must do before using a tool. It requires lawyers to evaluate whether information entered into generative AI may be disclosed to or accessed by people outside the firm. The assessment also extends to people inside the firm who would not adequately protect it. Confidentiality does not stop at the firm’s perimeter. Even when an attorney oversees the use of a tool for professional work, lawyers still need to check how it handles information.

The Opinion identifies a particular risk in self-learning tools. Information from one client’s representation could appear in responses to lawyers working on other matters. That is different from a stolen database or a vendor deliberately selling information. The risk comes from how some systems process and keep information, which may cross between matters that should stay separate. How much does a no-logging promise protect if information is disclosed while the user is interacting with the tool?

A policy alone cannot answer that question. A promise not to retain logs does not establish isolation within or between sessions, and a no-training commitment does not by itself describe every inference-time access path. The Opinion’s requirement to evaluate risks before input therefore calls for more than reading retention terms.

What inference architecture actually controls — and why privacy policies operate on the wrong layer

Inference is the processing of live prompts, uploaded documents and responses. In ordinary cloud-based LLM inference, the provider’s infrastructure processes that information in plaintext, making it accessible in principle to the operator or others with access to that infrastructure. Training is a separate question. A provider can promise never to train on a document while still processing its readable contents to answer a query. Zero-data-retention terms address storage, but they do not by themselves eliminate exposure during transmission or computation.

Trusted Execution Environments, or TEEs, address a different layer. In the architecture described here, inputs are decrypted and processed inside a hardware-isolated enclave, and the output is returned without exposing the prompt or intermediate state to the surrounding provider infrastructure. Remote attestation allows a user to cryptographically verify the model and the TEE’s security properties before submitting sensitive information.

The Talaria research illustrates both the possibilities and the limits. It identifies shortcomings in existing approaches, including differential privacy and standard confidential virtual machines, in comprehensively preventing leakage of model parameters and responses. Its framework partitions computation between a client-verified Confidential Virtual Machine and cloud GPU computation, using a Reversible Masked Outsourcing protocol to obscure intermediate data. It demonstrates that confidentiality of prompts and responses can coexist with lossless output quality. Confidentiality depends on the complete design behind a security label.

These mechanisms supply a stronger factual basis for confidentiality, but the legal analogy needs care. Provider-inaccessible processing differs from handing readable information to a commercial intermediary. It does not follow that the cited courts have adopted enclaves as a privilege requirement or safe harbor. Architecture can strengthen the confidentiality element without supplying the attorney relationship, legal purpose or litigation context that the relevant doctrine also requires.

Can enterprise AI tools that do not train on client data be enough?

The strongest objection is that professional confidentiality has never required eliminating every third-party service. Document management systems, email platforms and legal research databases use outside servers, and privilege usually survives that use. The law accommodates trusted professional intermediaries with appropriate contractual relationships. Enterprise AI agreements belong in that analysis. Zero-retention options, defined confidentiality duties and explicit training prohibitions provide a materially better factual record than consumer terms permitting broad reuse and disclosure.

It is therefore too strong to say that contracts are irrelevant or enterprise AI necessarily defeats privilege. The cited courts have not ruled that enterprise use destroys protection. Contracts set out what vendors may do and help give users a reasonable expectation of confidentiality. They do not establish every technical fact. A breach, compelled disclosure or a failure to keep information isolated can expose it even when rules limit how a vendor may normally use it.

For buyers, the useful comparison is between levels of protection. Consumer-tier tools with broad collection and disclosure permissions offer a different confidentiality basis from enterprise services with no-training and retention restrictions. Enterprise services, in turn, differ from architectures that also prevent provider access to plaintext during inference. These are distinct guarantees. For the highest-sensitivity work, the notes support combining contractual protections, attorney direction and documented governance with provider-inaccessible processing.

What organizations handling privileged documents need to assess before the next AI interaction

Organizations should assess attorney direction, platform architecture and documented governance before entering privileged information. These assessment categories apply differently across protection doctrines. Warner makes that qualification unavoidable because work product can exist without attorney direction. Still, when AI is used for substantive work in litigation, regulatory investigations or significant transactions, legal departments should require clear written direction from qualified counsel. The record should also show how outputs enter counsel’s professional work.

Does the service collect prompts and outputs, permit training, or allow disclosure to third parties or government authorities without a subpoena? Does its infrastructure process inputs in plaintext? For the highest-sensitivity work, the decisive technical question is whether the provider can access the information at all.

Written policies should say which platforms may be used, what kinds of information may be entered and what record of attorney direction must be kept. Select federal courts have begun issuing AI-specific protective orders barring corporate litigants from entering confidential case or business information without defined contractual safeguards. Governance must account for those restrictions. Trade secrets raise a parallel concern because voluntarily entering them into a service that permits vendor access and training is difficult to characterize as a reasonable step to preserve secrecy.

Organizations need a defensible record of the professional relationship, the platform’s processing boundaries and the rules governing its use. That record cannot guarantee a privilege ruling, but it addresses the facts a court will have to examine. Building it before the next interaction is more useful than trying to reconstruct confidentiality after a discovery dispute begins.