Supply Chain Risk in Open Model Weights

For most of the last decade, downloading a model checkpoint and loading it was equivalent to downloading a script and running it. The format that made that true is still in circulation, and the fix is worth understanding rather than just enabling.

What you are trusting when you load weights

A published model is a supply-chain artefact exactly like an npm package, and it carries three distinct trust questions that people tend to collapse into one:

• Does loading it execute code? A format question, with a clean answer.
• Are these the weights the publisher published? An integrity question, answered by hashes and signatures.
• Do these weights behave as advertised? A behavioural question, which no format and no hash can answer — a backdoored fine-tune has a perfectly valid checksum.

OWASP separates these as LLM03, Supply Chain, and LLM04, Data and Model Poisoning, and the separation is useful: the first two are solved engineering, the third is evaluation.

Whether any of this is yours depends on where inference runs. Calling a hosted API means the provider owns the loading question entirely and you inherit the third one — you are still trusting weights you cannot inspect, and your control is evaluation plus a contract. Downloading weights and serving them yourself moves all three onto your side of the line, and it is the case this page is about. The middle ground, where a platform runs an open model on your behalf, is the one to check explicitly: someone is loading a checkpoint, and it is worth knowing from where and in what format.

Pickle is a program, not a file format

PyTorch checkpoints were serialised with Python’s pickle, and pickle does not describe data — it describes a sequence of operations for a virtual machine that reconstructs objects. Its extension mechanism lets an object specify a callable and arguments to be invoked at load time. Deserialising is therefore executing, by design, and the Python documentation has always said so plainly: never unpickle data from an untrusted source.

The consequence for models is direct. A .bin or .pt file from a public hub could run arbitrary code the moment torch.load touched it — before any inference, with the privileges of the loading process, on a machine that by definition has GPUs and often has cloud credentials. Scanning helps and does not settle it: the callable can be resolved dynamically, so a scanner is pattern-matching against an expressible space it cannot enumerate.

What safetensors fixed

The safetensors format, from Hugging Face, is deliberately not a programming language. A file is a JSON header giving each tensor a name, dtype, shape and byte range, followed by the raw tensor bytes. Loading is parsing a header and memory-mapping a buffer. There is no callable to invoke, so the code-execution class is closed by construction rather than by detection.

Two secondary benefits made adoption easy: memory-mapped loading is fast and avoids a full copy, and the format is framework-neutral. The ecosystem moved accordingly, and PyTorch itself changed the default of torch.load to weights_only=True in version 2.6 (2025), restricting deserialisation to a safe subset. If you maintain code that pins an older PyTorch, that default is not doing anything for you.

# Prefer the format that cannot execute.
from safetensors.torch import load_file
state = load_file("model.safetensors")

# If a pickle checkpoint is unavoidable, constrain deserialisation
# explicitly rather than relying on the installed torch default.
state = torch.load("model.bin", weights_only=True, map_location="cpu")

# And treat remote-code flags as what they are: running someone else's
# Python. trust_remote_code=True is a code-execution decision, not a
# convenience flag -- pin a revision and review the code first.
model = AutoModel.from_pretrained(repo, revision=PINNED_SHA, trust_remote_code=False)

What the format cannot fix

• Backdoored weights. A model fine-tuned to behave differently on a trigger phrase is a well-formed safetensors file. The defence is provenance and evaluation, not parsing. See training data poisoning.
• Repository takeover and typosquatting. A name one character from the real one, or a compromised maintainer account, is the same attack the package ecosystems have. Pin a revision hash, not a tag or a branch.
• The surrounding files. Tokeniser configs, chat templates and custom modelling code ship alongside the weights. A chat template is prompt content you did not write, and trust_remote_code=True is straightforwardly arbitrary code execution with a friendlier name.
• Adapters and merges. A LoRA adapter modifies behaviour and is published far more casually than a base model. Inventory adapters like dependencies.
• Conversion steps. Quantised community rebuilds of a popular model are a re-publication by a third party. The original publisher’s reputation does not transfer to them.

A practical checklist

• Load safetensors where it exists; otherwise pass weights_only=True explicitly.
• Pin by commit hash. A tag can be moved; a branch certainly will be.
• Verify published checksums, and record the hash you loaded next to the version you served.
• Treat trust_remote_code as a code review, with a named approver.
• Fetch weights through an internal mirror, so an upstream deletion or replacement does not reach production unreviewed.
• Load and convert in a sandbox with no credentials and no network egress, then promote the artefact.
• Keep an inventory: base models, adapters, tokenisers, versions, hashes and who approved each.
• Run your own behavioural evaluation on any weights you did not train, including the quantised rebuild rather than only the original.

None of this is novel security engineering, which is the point. The model ecosystem gets safer by adopting the practices the package ecosystems learned the hard way, and the format change removed the one part that had no equivalent elsewhere.

The item people postpone is the inventory, and it is the one that decides how a bad day goes. When a popular repository is found to have been serving replaced weights, the only question that matters is which of your services loaded them and when — and that is a lookup if you recorded model, revision hash and load time per deployment, and a multi-day archaeology exercise if you did not. Treat it as the same obligation as a software bill of materials, because it is one.