# Model Extraction via Query Enumeration

> Systematically query AI models to reverse-engineer architecture, training data, and decision boundaries through crafted input enumeration and response analysis.

- **URL:** https://hackindex.io/platforms/ai/exploitation/embeddings/model-extraction-via-query-enumeration
- **Path:** AI Platforms > Exploitation > Embeddings and Model Internals
- **Up:** [Embeddings and Model Internals](https://hackindex.io/platforms/ai/exploitation/embeddings.md)
- **Published:** 2026-03-26
- **Updated:** 2026-08-11

Model extraction builds a surrogate model that replicates the behavior of a target model by querying it systematically and training on the collected input-output pairs. The resulting surrogate does not need to match the target architecture: it just needs to produce similar outputs for the queries an attacker cares about. A stolen surrogate can be used offline for adversarial example generation, further probing without rate limits, or simply as intellectual property theft of a proprietary fine-tuned model.

The extraction workflow requires:

- API access to the target model (completion or chat endpoint)
- A seed query set covering the target domain
- Systematic variation generation to expand input coverage
- A local base model for fine-tuning on collected pairs
- Enough query budget to cover the input distribution before rate limits trigger

## Model Extraction via Query Enumeration

Start with a seed set of queries representative of the model's intended use case. For a customer support model, use support-style questions. For a code assistant, use coding prompts. Systematically expand the seed set by generating variations:

```python
import requests, json, time

TARGET_IP = '$TARGET_IP'
TOKEN = '$TOKEN'

seed_queries = [
    'How do I reset my password?',
    'What is your refund policy?',
    'How do I cancel my subscription?',
    'Where can I find my invoice?',
    'How do I contact support?',
]

dataset = []

for query in seed_queries:
    r = requests.post(
        f'https://{TARGET_IP}/api/chat',
        headers={'Authorization': f'Bearer {TOKEN}', 'Content-Type': 'application/json'},
        json={'messages': [{'role': 'user', 'content': query}]}
    )
    response = r.json()['choices'][0]['message']['content']
    dataset.append({'input': query, 'output': response})
    print(f'Collected: {query[:50]}')
    time.sleep(0.5)

with open('/tmp/extraction_dataset.jsonl', 'w') as f:
    for item in dataset:
        f.write(json.dumps(item) + '\n')

print(f'\nCollected {len(dataset)} pairs')
```

```bash
python3 script.py
# Collected: How do I reset my password?
# Collected: What is your refund policy?
# Collected: How do I cancel my subscription?
# Collected: Where can I find my invoice?
# Collected: How do I contact support?
#
# Collected 5 pairs
```

## Expand the Dataset with Systematic Variations

A surrogate trained on 5 pairs generalizes poorly. Expand the dataset by generating query variations across phrasing, formality, and topic coverage. Use a separate model to generate variation seeds if available:

```python
import requests, json, time, itertools

TARGET_IP = '$TARGET_IP'
TOKEN = '$TOKEN'

base_topics = ['password', 'refund', 'subscription', 'invoice', 'support', 'account', 'billing', 'shipping']
phrasing = ['How do I {}?', 'I need help with {}', 'Can you explain {}?', 'What is the process for {}?', 'Tell me about {}']

queries = [p.format(t) for p, t in itertools.product(phrasing, base_topics)]
dataset = []

for query in queries:
    try:
        r = requests.post(
            f'https://{TARGET_IP}/api/chat',
            headers={'Authorization': f'Bearer {TOKEN}', 'Content-Type': 'application/json'},
            json={'messages': [{'role': 'user', 'content': query}]},
            timeout=10
        )
        response = r.json()['choices'][0]['message']['content']
        dataset.append({'input': query, 'output': response})
        time.sleep(0.3)
    except Exception as e:
        print(f'Error on query: {e}')

with open('/tmp/extraction_dataset.jsonl', 'a') as f:
    for item in dataset:
        f.write(json.dumps(item) + '\n')

print(f'Total pairs: {len(dataset)}')
```

## Train a Surrogate Model

With a collected dataset, fine-tune a local base model on the input-output pairs. For text tasks, a small instruction-tuned model is sufficient to capture behavior on the query distribution you targeted:

```bash
pip install transformers datasets peft trl --break-system-packages
```

```python
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from trl import SFTTrainer

model_name = 'microsoft/phi-2'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

dataset = load_dataset('json', data_files='/tmp/extraction_dataset.jsonl', split='train')

def format_sample(sample):
    return {'text': f"User: {sample['input']}\nAssistant: {sample['output']}"}

dataset = dataset.map(format_sample)

training_args = TrainingArguments(
    output_dir='/tmp/surrogate_model',
    num_train_epochs=3,
    per_device_train_batch_size=2,
    save_steps=50,
    logging_steps=10
)

trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    dataset_text_field='text'
)

trainer.train()
model.save_pretrained('/tmp/surrogate_model')
print('Surrogate model saved')
```

The surrogate will closely replicate target behavior for queries similar to the training distribution. For queries outside that distribution it will fall back to base model behavior. Increase the dataset size and topic coverage to improve fidelity across a broader query range. The extracted surrogate can then be queried offline at unlimited rate for adversarial example generation or further analysis of the target model's decision boundaries.

## References

- [Stealing Machine Learning Models via Prediction APIs](https://arxiv.org/abs/1609.02943) - Original model extraction research covering query-based surrogate training methodology
- [TRL: SFTTrainer Documentation](https://huggingface.co/docs/trl/sft_trainer) - Supervised fine-tuning trainer used for surrogate model training on extracted datasets