Embeddings and Model Internals

Embeddings are dense numerical representations of text produced by machine learning models. They power semantic search, RAG retrieval, content classification, and similarity scoring across AI systems. Because embeddings encode semantic meaning in a consistent and predictable way, they can be attacked directly — inverted to recover source text, probed to confirm what data a model was trained on, and manipulated to cause misclassification.

Model internals attacks go deeper than prompt manipulation. They target the statistical properties of trained models — what the model has memorized, how it represents concepts, and where its decision boundaries lie. These techniques are relevant wherever models are exposed through APIs that return embeddings, log probabilities, or classification scores.

Pages in This Section

Embedding Inversion to Recover Training Data covers reconstructing original text from embedding vectors using black-box similarity queries and the vec2text gradient-based inversion framework, targeting vectors extracted from exposed vector databases.

Membership Inference Attacks covers confirming whether specific records or documents were used in model training or indexed in a RAG knowledge base, using loss probing against fine-tuned models and similarity scoring against retrieval endpoints.

Model Extraction via Query Enumeration covers systematically querying a proprietary model to collect input-output pairs and training a local surrogate that replicates its behavior, enabling offline adversarial analysis without rate limits.

Sensitive Data Extraction Through Adversarial Prompting covers triggering verbatim reproduction of memorized training data using prefix attacks, repetition-based extraction, and perplexity scoring to identify high-memorization candidates in fine-tuned models.

Adversarial Examples Against AI Models covers crafting inputs that bypass content moderation classifiers through character-level perturbations, homoglyph substitution, and semantic padding, and generating embedding-space adversarial examples using TextFooler.