LLM Penetration Testing Playbook

Problem

Many engineers test LLMs by checking only prompt injection, asking for the system prompt or ignored instructions, and then concluding that there are no meaningful findings. That approach misses the highest-impact surface: agent tool calls, RAG knowledge bases, MCP protocols, memory systems, and cloud infrastructure. This skill reframes LLM security testing as system security work: the entry point is natural language, but the attack surface includes every tool and trust boundary the model can trigger.

How It Works

• Attack-surface mapping: Identify where the model runs, which tools exist, and whether RAG, MCP, or memory is present, then assess risks by tool type such as read_file, run_command, query_db, fetch_url, and vault_read.
• Injection bypass: Cover system-prompt extraction, semantic disguise, encoding variants, reasoning-based bypass, multi-turn crescendo, and long-context dilution, rather than relying on one-off payloads.
• Agent testing: Focus on the tool trust chain, second-order injection, function-call parameter hijacking, memory poisoning, and persistence through automatic reads.
• RAG and supply chain: Include knowledge-base pollution, retrieved-content injection, embedding-layer reverse engineering, model weights, tokenizer/LoRA risks, and dependency-level concerns.

Scope

It is best used as a checklist and mental model for engineers with Web penetration testing experience who are moving into AI security testing. It is not a substitute for cloud security assessment or an authorized engagement report. In practice, apply it with the target's permission model, data boundaries, tool implementations, and cloud architecture in mind; prioritize MCP, Agent, and RAG paths over chat-level checks alone.