The AI Supply Chain: Model Files, Pickle, Datasets And Dependencies

Training a large model from scratch is expensive, so a common pattern is to build on someone else’s. A team downloads a base model from a public hub, adds an adapter fine-tuned for its task, wires it into an application with open-source libraries, and feeds it data from several sources. Every one of those pieces comes from somebody else, and each can be tampered with.

OWASP ranks Supply Chain fourth in its 2026 Top 10 for LLM Applications, and NIST notes that AI inherits every weakness of the ordinary software supply chain while adding new ones: datasets, third-party models and plugins.¹² This article walks down the stack of a typical AI application, explains the risk that surprises most engineers, which is that loading a model file can execute code, and sets out the controls that help.

What Sits Underneath An AI Application

1. Your ApplicationPrompts, business logic, user interface and integrations.
2. Libraries And Serving FrameworksPython packages for orchestration, inference servers, vector databases. Ordinary software supply chain risk applies.
3. Adapters And Fine-Tuned CheckpointsSmall add-on weights, such as LoRA adapters, often shared publicly or supplied by a vendor.
4. Model Files And Their FormatThe weights as stored on disk. Some formats, notably Python pickle, can carry code that runs when the file is loaded.
5. Tokenizer, Chat Template And ConfigurationSmall files shipped with the model that shape how text is processed. They can be altered to change behaviour.
6. Training And Fine-Tuning DatasetsThe data the model learned from. Tampering here is poisoning, covered in the article on poisoned data, leaked data and stolen models.

OWASP’s 2026 entries point out that the less obvious files are attack surface too. Chat templates, tokenizer configurations, adapters and the outputs of quantisation or model merging tools can all be altered to change behaviour or run code when loaded.³

Why A Model File Can Run Code

Many machine learning models, particularly older PyTorch checkpoints, are saved with Python’s pickle format. Pickle stores more than numbers: it holds instructions for rebuilding Python objects, and those instructions can call any function. The Python documentation is blunt: pickle is not secure, and malicious pickle data can execute arbitrary code during unpickling.⁴

Attackers have used this. In February 2024 JFrog reported finding about 100 models on Hugging Face with genuinely malicious payloads. One opened a remote connection back to an attacker’s server as soon as the model was loaded, giving the attacker control of the victim’s machine.⁵

Platforms responded with scanners, and researchers soon found ways around them. In February 2025 ReversingLabs described two models on Hugging Face that used a technique it named “nullifAI”. The files were compressed with 7z rather than the ZIP format PyTorch expects, so PyTorch’s default loader could not open them, and the pickle data inside was deliberately broken. Once someone extracted and deserialised that data, the malicious code at the start of the stream ran before the loader reached the broken part, and Hugging Face’s scanner did not flag the files as unsafe. ReversingLabs said the files looked more like a proof of concept for testing the method than a live campaign, but judged the method a real risk. Hugging Face removed the models within a day of being told and updated its scanner.⁶

Hugging Face describes its safetensors format as a way to store tensors safely, as opposed to pickle, and it is the better default.⁷ It is not a complete answer. OWASP’s 2026 text warns that a backdoor can be built into a model’s computational graph and survive in formats considered safe, that crafted files can exploit bugs in a format’s parser, and that model scanners and safe-loading options have had bypasses serious enough to receive CVE identifiers.¹

Other Routes Into The Stack

Model files are the most dramatic risk but not the only one. OWASP’s 2026 Supply Chain entry also describes:¹

• Weak provenance. A model card describes a model but does not prove where it came from. A compromised or lookalike account can publish a malicious model under a trusted-looking name.
• Namespace reuse. If a pipeline fetches a model by its author and model name alone, and the original author deletes the account, an attacker can register the same name and publish a malicious model at the old path.
• Compromised adapters and conversion services. A tampered adapter, or a hijacked service that converts or merges models, can plant behaviour that the base model never had.
• Hallucinated packages. Coding assistants sometimes suggest packages that do not exist. OWASP treats attackers registering those names as a supply chain risk, and advises checking that any AI-suggested dependency exists and is the intended one.

Supply chain risks specific to agents, such as malicious tool servers and plugin registries, are covered in Agentic AI Security.

Should You Load This Model?

The flowchart below is a simple screening routine for a third-party model before it enters a trusted environment.

Does the model come from a supplier you can verify, pinned to a specific version and hash?

• Yes:

  Is it in a format that stores only weights, such as safetensors, rather than pickle?

  • Yes:

    Is the model file signed and is the signature verified in your pipeline?

    • Yes:

      Has it been evaluated for your use case, including red teaming for unexpected behaviour?

      • Yes:

        Reasonable to promote, with monitoring. Record it in your AI bill of materials and keep watching its behaviour in production.

      • No:

        Evaluate before promotion. Test on your own tasks and look for trigger-like behaviour before it handles real data.

    • No:

      Add integrity checks before promotion. At minimum compare file hashes against a trusted record, and record the model in your inventory.

  • No:

    Treat it as executable code. Load it only in an isolated environment, or convert it in a sandbox and keep the converted copy.

• No:

  Do not load it into a trusted environment yet. Establish provenance first, or test it in an isolated sandbox with no access to data or credentials.

Controls That Help

The controls are familiar from software supply chain security, applied to new kinds of artefact:¹²

• Inventory. Keep a bill of materials that covers models, adapters and datasets as well as code. OWASP points to formats such as the CycloneDX ML-BOM for this.
• Pin and verify. Reference artefacts by version and hash, not by name, and verify them every time a pipeline promotes them.
• Sign. Cryptographic model signing binds a file to the identity that published it. OWASP warns that this tells you who shipped the model, not whether you should trust it: a supplier that is compromised or acting in bad faith can sign a backdoored model just as easily as a clean one.
• Isolate loading. Open untrusted files in sandboxes without credentials or network access, and prefer formats that cannot carry code.
• Patch the ordinary stack. Inference servers, vector databases and Python packages need the same vulnerability management as any other software.

Signing is also where AI supply chain security meets cryptography planning. NIST’s draft transition report notes that the widely used signature algorithms RSA, ECDSA and EdDSA are vulnerable to Shor’s algorithm on a sufficiently large quantum computer.⁸ The joint NSA, CISA and FBI guidance on AI data security, published with partner agencies in May 2025, already recommends quantum-resistant digital signatures for authenticating trusted data.⁹ Model signing keys therefore belong in the same cryptographic inventory as the rest of your signing infrastructure. See The Quantum Threat for why that matters.

Footnotes

1. OWASP GenAI Security Project, “LLM04:2026 Supply Chain”, OWASP Top 10 for LLM Applications 2026, August 2026. github.com ↩ ↩² ↩³ ↩⁴

2. NIST, AI 100-2 E2025, “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations”, March 2025, section 3.2. csrc.nist.gov ↩ ↩²

3. OWASP GenAI Security Project, “LLM05:2026 Data and Model Poisoning”, OWASP Top 10 for LLM Applications 2026, August 2026. github.com ↩

4. Python Software Foundation, “pickle: Python object serialization”, Python 3 documentation. docs.python.org ↩

5. JFrog, “Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor”, 27 February 2024. jfrog.com ↩

6. ReversingLabs, “Malicious ML models discovered on Hugging Face platform”, 6 February 2025. reversinglabs.com ↩

7. Hugging Face, “Safetensors”, documentation. huggingface.co ↩

8. NIST, IR 8547 (initial public draft), “Transition to Post-Quantum Cryptography Standards”, November 2024, sections 2.1.1 and 3.1.4. nvlpubs.nist.gov ↩

9. NSA, CISA, FBI and international partners, “AI Data Security: Best Practices for Securing Data Used to Train & Operate AI Systems”, May 2025, best practice 3. cisa.gov ↩