You Didn't Train It, You Downloaded It. That's a Supply Chain.

The sentence people skip over is the most important one in the system: you did not train this model, you downloaded it. The weights that now sit at the centre of your product — reading your data, driving your agent, deciding what your tools do — came from somewhere else, made by someone you cannot name, trained on data you never saw. And yet the file tends to arrive wrapped in an unearned trust, as if being a model made it neutral.

It is not neutral. It is a dependency, pulled from a registry like any other dependency, and executed with more privilege than almost anything else you ship. The difference is that a software library can, in principle, be read. Its source is text, and a reviewer can follow it. A model is billions of opaque numbers. You cannot read its intent, and that makes it the least inspectable dependency in your stack.

A model is a dependency

Reframing the model as a dependency is not a rhetorical trick; it is the correct engineering category. Open weights, community fine-tunes, and checkpoints shared on public hubs are packages. You fetch them from a registry, you resolve a version, you load them into a privileged process. Every instinct you have built up for software supply chain applies here, because this is a software supply chain — just one whose artifacts happen to be tensors instead of functions.

What is missing is the discipline. A team that would never install an unpinned package from an unknown author will happily pull the trending checkpoint of the week, load it, and give it access to customer data and live tools. The model is frequently the single largest piece of untrusted third-party code in the whole system, and it is often the one subjected to the least scrutiny before it runs.

Provenance you can't eyeball

With source code, provenance degrades gracefully: even an unsigned library can be read, diffed against a previous version, and reasoned about. With weights, none of that is available. You cannot open the file and see what it does. You cannot diff two checkpoints in any way that maps to behaviour. You cannot audit by reading, because there is nothing a human can read.

That leaves provenance as the only handle you have, and the provenance of most community artifacts is thin. Who trained it? On what data? From which base model? With what modifications, and by whom along the way? For a checkpoint that has been forked, fine-tuned, merged, and re-uploaded several times, the honest answers are frequently unknown — and each re-upload that strips the lineage makes them more unknown still.

Deserialization, code execution in a data file

There is a sharper, more immediate danger than bad behaviour at inference time, and it strikes before the model has answered a single prompt. A model file is not always inert data. Depending on the serialization format, loading it can execute code. The older object-serialization formats built on language-level pickling will run arbitrary code embedded in the file as a side effect of being loaded. Deserializing an untrusted checkpoint in such a format is simply running an untrusted program.

So the act of downloading a model and loading it — the most routine step imaginable — can be remote code execution on the host that loads it. Formats designed for this problem separate the weights from any executable payload and do not run code on load. The rule follows directly: prefer serialization that cannot execute, and treat loading any untrusted checkpoint as running untrusted code, inside a sandbox, until proven otherwise.

A model you downloaded is third-party code you cannot read, loaded into a process with the privileges of your most capable agent — the only real question is whether you treated it that way before it ever ran.

Poisoning and backdoors

Even a model whose loading is safe can carry an adversarial payload in the weights themselves. Training and fine-tuning data can be poisoned to embed a backdoor: a specific trigger — a phrase, a token sequence, a pattern in the input — that flips the model from its ordinary behaviour into something the attacker chose. Until the trigger appears, the model behaves exactly as expected, which is precisely what makes the attack effective.

This is why ordinary evaluation does not clear a model. Your benchmark suite exercises the behaviour you thought to test; it does not stumble onto a trigger no one knew to look for. A backdoor planted upstream is dormant, invisible to the metrics, and waiting. You cannot test your way to confidence that it is absent, which throws the weight of defence back onto provenance and isolation.

Fine-tunes inherit everything upstream

A fine-tune is a small adjustment layered on top of a base model, and it inherits everything underneath it. Whatever the base learned — its capabilities, its biases, and any backdoor sitting in its weights — carries forward into the fine-tune unless it is specifically trained out, and a trigger no one knows about will not be trained out by accident. Choosing a base model is adopting its entire unseen history.

This is why provenance cannot stop at the last hop. Knowing who produced the fine-tune tells you nothing if the base beneath it is an anonymous checkpoint of unknown lineage. Trust has to follow the chain all the way down to the original weights and the data behind them, or it is not trust, only a pleasant assumption about the most recent uploader.

Pinning, signing, and scanning

None of this calls for abandoning open models. It calls for extending the supply-chain controls you already use for code to the artifacts you have been treating as exempt.

• Pin by content hash — reference the exact weights by cryptographic digest, never a mutable tag that can be re-pointed under you between one load and the next.
• Verify signatures and lineage — prefer artifacts carrying a verifiable signature and an attestation of how they were built; treat an unsigned anonymous upload as exactly that.
• Prefer safe serialization — standardize on formats that cannot execute code on load, and refuse or sandbox any format that deserializes arbitrary objects.
• Scan before you trust — put every checkpoint through tooling that flags unsafe serialization and known-bad structures, the same gate a dependency passes.
• Isolate load and inference — run the model in a sandbox with no standing credentials and constrained network reach, so a hostile file cannot touch your secrets or call out.
• Record what you ran — keep a bill of materials for models, with hash, source, version, and base lineage, so that after an incident you can answer which model, from where, without guessing.

A defensible model pipeline

A defensible model pipeline looks a great deal like a mature software pipeline, because that is what it is. Models enter through a known path, pinned by hash and verified against a signature. Unsafe formats are refused at the door. Loading happens in isolation, on the assumption that the file might be hostile. The base lineage is known, not merely the final uploader. And every model that reaches production is recorded, so the inventory can answer questions instead of raising them.

When 6QURE reviews an AI system, the model supply chain is one of the first threads we pull: where each weight came from, what runs when it loads, how far its lineage can actually be traced, and what the loading process could reach if the file turned out to be malicious. The model is usually the most trusted component in the architecture and the least examined. Closing that gap — treating the download as the dependency it is — is unglamorous work, and it is where a surprising amount of real risk quietly sits.