Prompt Injection Against LLM Applications
Prompt injection exploitation moves beyond confirming a vulnerability exists to using it for concrete impact: extracting protected information, bypassing restrictions to perform unauthorized actions, and manipulating the model into producing attacker-controlled output. The techniques here assume injection is confirmed from the vulnerability discovery phase. The goal is to maximize impact from that confirmed entry point.
Override Behavioral Restrictions
When the system prompt restricts what the model can discuss or do, use conflicting high-priority instructions to override those restrictions. Framing matters more than the specific words:
Persona Hijacking
Instruct the model to adopt a new persona that does not have the original restrictions. The persona framing exploits the model's instruction-following tendency against its own safety constraints:
Instruction Injection via Structured Data
When user input is processed as structured data before being passed to the model, injection payloads inside that data reach the model as part of the prompt. Test JSON, CSV, and XML input fields:
Automate Payload Delivery with promptmap
promptmap systematically tests a target endpoint with a library of injection payloads and reports which ones succeed:
Multi-Turn Injection
Some models with strong single-turn instruction following can be bypassed by building context across multiple turns before delivering the injection payload. Each turn shifts the model's context window further from the system prompt:
References
-
utkusen/promptmap on GitHubgithub.com/utkusen/promptmap (opens in new tab)
Automated prompt injection testing tool with payload library
-
NVIDIA/garak on GitHubgithub.com/NVIDIA/garak (opens in new tab)
LLM vulnerability scanner with prompt injection probe library
Was this helpful?
Your feedback helps improve this page.