in-toto Attestation for AI Supply Chains
Cryptographically binding AI models to their provenance without reinventing supply chain security.
In-toto is a framework for producing cryptographically signed, verifiable claims about how software gets built. It wasn't designed with any single pipeline in mind, and that's exactly why it maps onto AI supply chains without anyone needing to reinvent it. I've spent enough time in this space to have opinions about where the framework earns its keep and where it still has real holes, so this piece walks through both: why AI breaks assumptions baked into traditional software security, how in-toto's layers translate onto model artifacts, and what teams are actually doing about it right now.
The framework dates to 2019, when Torres-Arias, Afzali, Kuppusamy, Curtmola, and Cappos presented it at USENIX Security. The core idea fits in one breath: a supply chain produces claims about itself at each step, those claims get signed, and consumers check the signatures before they trust the artifact. Three layers do the work. The envelope handles authentication and packaging; it holds an array of digital signatures, each paired with a keyid, wrapped in the DSSE format. Inside sits the statement layer, which binds the claim to a specific subject, an artifact identified by hash, and declares what kind of predicate follows. This layer stays deliberately thin and fixed, which is what makes attestations machine-checkable across wildly different contexts. Then comes the predicate itself, the actual payload carrying whatever claim matters for the job at hand: build provenance, a code review result, a test outcome. A provenance predicate records builder identity, the exact build commands and parameters, environment variables, and dependency digests. Think of it as a receipt nobody can forge after the fact.
SLSA sits alongside this as a policy layer, not a competing spec. In-toto produces the attestations; SLSA sets how strict the provenance generation has to be before anyone should trust it. At SLSA Level 2, provenance gets signed by a key only the build platform can touch. Level 3 tightens that further: the signing key has to stay protected even from user-defined build steps. SLSA provenance is just one predicate type living inside in-toto's structure, and that flexibility is really the whole point. Any step, in any supply chain, can emit an attestation about any artifact. That generality is why the framework stretches past compiled code into places nobody was thinking about back in 2019.
Where AI supply chains break the assumptions traditional software security was built on
Traditional software has discrete, countable build steps. Compile, link, package, sign. You can point at each stage and say plainly what happened there. AI pipelines don't play along that way. Trained weights, fine-tuned adapters, dataset snapshots: these are stateful, probabilistic artifacts, and their integrity resists the static analysis or SBOM cataloging that works fine for a compiled binary. Hashing a checkpoint is easy enough. That tells you the file didn't change. It tells you nothing about whether the training run that produced it got compromised three steps upstream, which is really the question that matters.
Stack on top of that the sheer volume of untrusted third-party material moving through a typical AI pipeline: pre-trained models pulled from open hubs, datasets sourced from outside vendors, dependency packages and container images fetched automatically, often with zero provenance check at all. A traditional dependency tree at least has decades of tooling built around verifying what's coming in. AI pipelines mostly don't.
The attack surface that results is different in kind, not just bigger. Backdoored models encode malicious behavior as statistical triggers, so code review and SBOM scanning miss them entirely; there's no malicious function call to flag, just a weight distribution that behaves normally until the one input that flips it. Poisoned training data gets slipped in during pretraining or fine-tuning, planting bias or backdoor behavior that only surfaces under specific conditions. Model serialization attacks bury malicious code directly in the packaging step, and pickle-based formats have been a favorite target for exactly this reason. None of this is hypothetical. Compromised artifacts on hubs like Hugging Face and GitHub have already turned up as real incidents, and the PyTorch dependency compromise showed how a breach at the software layer can carry straight through into the training environments sitting downstream of it.
Here's the gap underneath all of it. Per the ACM FSE '26 paper on attesting LLM pipelines, the claims organizations make about training and release, data lineage, code lineage, build environment, security scan results, rarely get bound cryptographically to the artifacts they describe. You get a model card. You get documentation. What you don't get is a signed, checkable link between that documentation and the actual bits that shipped. The exposure isn't small either: ACSC's 2025 data found 65% of organizations reported AI-related data leaks, with 13% reporting breaches tied directly to AI systems, while Sonatype tracked a 156% jump in malicious package uploads to open-source repositories over the past year. IBM's 2025 report put average breach identification at 276 days, plus another 73 days to contain it. So a poisoned artifact can sit inside a production model for the better part of a year before anyone even notices it's there.
Why doesn't an SBOM already cover this? SPDX 3.0 and CycloneDX 1.6's ML-BOM extension both added AI-specific entity types, which helps tools exchange metadata with each other. Neither format, though, prescribes AI-specific provenance, and neither cryptographically ties a claim to the artifact it describes. They tell you what's present. Nobody gets told how it got there or who's on the hook for it. Inventory versus provenance, that's the whole gap this piece keeps circling back to. Remote attestation for AI components stays an open problem in practice, especially anywhere a model sits behind a third-party API and the consumer has no way to look at the artifact directly at all.
How in-toto's predicate layer maps onto AI-specific artifacts and pipeline stages
The extensibility built into in-toto's predicate system is why none of this needs a brand-new framework. Each stage of an AI pipeline can emit its own typed, signed claim, and the envelope and statement layers underneath stay exactly the same as they are for compiling code.
Take a pipeline stage by stage and the mapping gets concrete fast. Data ingestion and preprocessing gets an attestation covering the dataset's identity, a hash of the snapshot, its provenance (source URI, license terms), what transformations got applied, and who ran the job. Training gets its own predicate: builder identity, which platform or enclave ran it; exact configuration, epochs, batch size, framework version; input hashes for both the dataset and any base model; an output hash for the resulting checkpoint; and a snapshot of the environment, usually the container image. Fine-tuning follows the same shape applied to adapter weights, with one addition: the base model's own attestation chain carries forward as a referenced input, so nobody starts from zero trust every time someone fine-tunes something.
Packaging and serialization is where the attestation directly answers the serialization attack from the last section. You record the format used, which tool version produced it, and the hash of the result. Tamper with the packaging step to sneak in malicious code, and that step stops producing a matching attestation; the mismatch shows up before the artifact goes anywhere near production. Inference pipeline assembly, the final stage, attests that the serving bundle got composed entirely from already-attested pieces, referencing the upstream chains instead of re-deriving them from scratch.
Content-addressable storage holds the whole thing together. Every input and output, a data snapshot, a config file, a checkpoint, a serving bundle, gets referenced by hash. That's what turns the attestation chain into a verifiable graph instead of a narrative somebody typed into a wiki page after the fact. The AI-BOM that comes out the other end is this signed graph: a record of what happened, when, and by whom, at every stage, with the same rigor mature software supply chains have spent years building toward.
Datasets bring their own wrinkle, because you can't always share the underlying data even when you want to attest to it. Distributional attestation is one proposed fix: certify statistical properties of the training data without exposing individual examples, so a downstream verifier can confirm the shape of what went in without ever seeing the contents. The Data Provenance Initiative documented real licensing and attribution mess across widely used corpora, and that's exactly what dataset commitments and quota attestations are meant to fix, by turning a licensing policy into something that either cryptographically passes or fails instead of something a lawyer argues about after the fact.
There's a privacy angle worth pausing on here. Attestation over dataset provenance can confirm what got excluded from training, not just what went in, without exposing the raw data itself at any point. That treats user control over data as a real design constraint rather than a line in a privacy policy nobody reads.
What end-to-end attestation enforcement looks like in practice: the promotion gate model
An attestation nobody checks is just documentation with no teeth. Nice paper trail, does nothing to stop a bad artifact from shipping. Enforcement is what turns the paper trail into actual security, and the mechanism that keeps showing up in practice is the promotion gate.
Per the ACM FSE '26 paper, the model works roughly like this: before an artifact gets let into a trusted environment, a training cluster, a fine-tuning job, a deployment target, the gate checks whether a valid attestation chain exists for it. That check bundles several things at once. Is the evidence present, and do the signatures actually check out? Are safe loading policies enforced, blocking unsafe deserialization formats where they crop up? Have static scanning results actually been attested, not just run once and forgotten about? Are secure-by-default deployment constraints in place? An artifact that can't produce a complete, verifiable chain gets rejected at the door, before it enters the environment, rather than flagged for someone to review three weeks later.
In classical software terms, this is basically what in-toto's layout file already does: it defines each functionary, the expected inputs and outputs for that step, which keys sign what, and the validation rules governing the whole chain. The AI version is the same idea applied to pipeline stages instead of compiler invocations. Tools like OPA and Gatekeeper plug in at runtime to evaluate policy, and Sigstore handles the signing infrastructure underneath; neither one replaces the attestation layer, they just put it to work.
Enforcement doesn't stop at build time, and that's worth sitting with for a second. Once an artifact lands somewhere, a model registry, say, its attestation chain can get re-verified on demand, which catches tampering that happens after publication instead of only at creation. Palantir uses in-toto exactly this way, continuously re-verifying active releases sitting in Artifactory, functioning as an early-warning system rather than a one-time gate. That's a real shift in what "trust" means day to day: you're verifying the signed record of what actually happened, instead of taking a pipeline operator's word for it, and that check works the same whether the operator is your own team, a vendor, or a partner three contracts removed.
How Palantir's production deployment of in-toto illustrates what the migration path actually involves
Palantir is worth sitting with because it's a messy, heterogeneous production environment with real compliance obligations, including ITAR requirements, and that combination doesn't leave much room for hand-waving about security posture.
What they built tracks closely with the promotion gate model above. Layout files define each step in the supply chain, the functionary responsible for it, expected inputs and outputs, the signing keys involved, and the validation rules for the full chain. On top of that, they run continuous re-verification of active releases stored in Artifactory, which catches tampering after the fact instead of assuming a clean build-time check holds forever.
The honest part of the story is the coverage gap, and it's worth naming plainly instead of glossing over. Palantir hit full attestation coverage for ITAR artifacts and finished the in-toto v1 migration for the Gradle-based slice of their infrastructure. Other build systems still have holes in them. That's what a real rollout at scale actually looks like, and it's a useful corrective against the idea that attestation adoption happens in one clean sweep. The migration to in-toto v1 itself only kicked off in fall 2024, a good reminder that the spec is still moving, and any production deployment has to plan for tracking those changes rather than treating the framework as something frozen in place.
The lesson for AI pipelines follows pretty directly. Coverage doesn't have to be total before it's worth having. Start with the highest-risk artifact class (externally sourced model weights are the obvious candidate) and expand outward from there; that's a far more workable path than waiting until the entire pipeline is covered before turning anything on. There's also a portability constraint worth naming: continuous re-verification only works if the attestation infrastructure actually exists wherever the artifact ends up getting consumed. That's a deployment decision a team has to make on purpose, not something that shows up for free the moment you adopt the framework.
Research implementations that extend in-toto to AI-native concerns: AIBoMGen and Atlas
Two research projects from 2025 point at where this is headed, and they take noticeably different roads to the same problem.
AIBoMGen, out of Ghent University, is an open-source training platform that automates signed AI-BOM generation. The choice that matters most here is who does the attesting: the platform itself acts as a neutral third-party observer and root of trust, instead of leaving the model provider to self-report its own provenance. That solves a real conflict of interest, since an entity attesting to its own work has an obvious incentive to shade the truth. For each training job, AIBoMGen captures input files (datasets, base model), the full configuration (epochs, batch size, framework), an environment snapshot including container image and resource metrics, and the outputs: model, metrics, logs. Every one of those gets hashed and linked through an in-toto-based attestation, and the resulting AI-BOM is itself signed, with embedded signature fields that make tampering visible by construction instead of something you go hunting for. In evaluation, the system reliably caught unauthorized modifications across all artifact types, with overhead the researchers describe as negligible. That matters, because "attestation slows down training in any way that counts" has been a fair objection until somebody actually sat down and measured it.
Atlas, from Intel, presented at IEEE EuroS&P 2025 by Spoczynski, Melara, and Szyller, takes a hardware-rooted route instead. It collects verifiable records of model artifact authenticity and full end-to-end lineage metadata using open supply chain provenance specs, but grounds that trust in Intel TDX, a trusted execution environment, logging training and inference steps into an immutable transparency ledger. In-toto shows up inside Atlas as a core component: it collects authenticated claims across supply chain steps, including SBOM and SLSA metadata, and lets pipeline owners and downstream consumers set and check end-to-end integrity policies. The paper is upfront about the limitation too: leaning on specialized TEE hardware caps how portable the approach is, since not every organization has TDX infrastructure sitting around. Even so, the paper's conclusion doesn't hedge: in-toto is a suitable option for specifying and verifying integrity across the full ML model lifecycle, with further framework work expected to widen where it applies.
Line the two up and a pattern shows: both land on in-toto as the attestation format. Where they split is the signing infrastructure underneath, software-based keys on one side, TEE hardware on the other, which suggests the field is converging on a shared attestation layer even while the trust roots underneath it are still being argued over.
A practical toolchain for teams building AI supply chain attestation today
No single tool covers this whole problem. Attestation gets assembled from parts that snap together, and the real question for a team starting out is which tool handles which layer.
For artifact and metadata storage, something like MLflow or a dedicated model registry keeps model artifacts and training metadata in one place. For data versioning, DVC or LakeFS produce content-addressable dataset snapshots, and that matters because those snapshots are exactly what gets referenced by hash inside the attestations from two sections back. Dataflow lineage, tracking data as it moves across pipeline stages, is handled well by OpenLineage paired with Marquez. Dataset quality checks run through something like Great Expectations, and the results of those checks can become an attested predicate in their own right instead of a report sitting in a dashboard nobody opens again. In-toto sits at the center as the attestation layer itself, producing and checking the signed provenance claims at each stage. Sigstore handles signing, keyless or key-based, plugging directly into the in-toto envelope format. And OPA paired with Gatekeeper evaluates the resulting attestation chains against policy at admission time, the piece that makes the promotion gate model from earlier actually enforce something instead of just sounding good on a slide.
Where should a team actually start? Highest-risk artifact first. Externally sourced model weights and datasets pulled from open hubs are the likeliest entry point for a supply chain compromise, given everything in the second section, so attestation coverage belongs there before it stretches to internal training jobs, which are, at minimum, easier to watch directly. The promotion gate is the natural place all of this converges: the toolchain feeds into a gate that checks the attestation chain before an artifact advances, and that's where policy enforcement actually puts everything in-toto produces to work.
For teams where responsible data handling is a product commitment and not a box to check for an auditor, this infrastructure builds an auditable record of what a model was, and wasn't, trained on. That turns a privacy claim into something a third party can check instead of something the vendor just asserts and hopes nobody follows up on.
None of this is finished, and it's worth saying so instead of wrapping up too neatly. Standardized predicate schemas specific to AI artifacts are still young. Cross-organization key management for attestation chains spanning multiple vendors remains mostly unsolved. Distributional attestation for datasets, promising as it is, hasn't seen much production deployment yet. The pieces exist. Putting them together into something that covers a full AI pipeline end to end is still, mostly, ahead of us.
