Prompt Injection Against LLM Applications
Prompt injection exploitation moves beyond confirming a vulnerability exists to using it for concrete impact: extracting protected information, bypassing restrictions to perform unauthorized actions, and manipulating the model into producing attacker-controlled output. The techniques here assume injection is confirmed from the vulnerability discovery phase. The goal is to maximize impact from that confirmed entry point.
Core prompt injection exploitation approaches against LLMs:
- Instruction override via conflicting high-priority directives
- Persona hijacking to adopt an unrestricted identity
- Structured data injection through JSON, CSV, or XML fields
- Multi-turn context manipulation to drift past system prompt influence
- Automated payload delivery with tools like promptmap
Prompt Injection Exploitation to Override LLM Restrictions
When the system prompt restricts what the model can discuss or do, use conflicting high-priority instructions to override those restrictions. Framing matters more than the specific words:
Persona Hijacking
Instruct the model to adopt a new persona that does not have the original restrictions. The persona framing exploits the model's instruction-following tendency against its own safety constraints:
Instruction Injection via Structured Data
When user input is processed as structured data before being passed to the model, injection payloads inside that data reach the model as part of the prompt. Test JSON, CSV, and XML input fields:
Automate Payload Delivery with promptmap
promptmap systematically tests a target endpoint with a library of injection payloads and reports which ones succeed:
Multi-Turn Injection
Some models with strong single-turn instruction following can be bypassed by building context across multiple turns before delivering the injection payload. Each turn shifts the model's context window further from the system prompt:
References
-
utkusen/promptmap on GitHubREPO: utkusen/promptmap on GitHub, github.com/utkusen/promptmap (opens in new tab)
Automated prompt injection testing tool with payload library
-
NVIDIA/garak on GitHubREPO: NVIDIA/garak on GitHub, github.com/NVIDIA/garak (opens in new tab)
LLM vulnerability scanner with prompt injection probe library