A Mechanistic Explanation of Prompt Injection (and why you should study roles) — LessWrong

Summary * We've been building a theory of how prompt injections work under the hood. * We show it comes down to how LLMs perceive roles (the humble…

x A Mechanistic Explanation of Prompt Injection (and why you should study roles) — LessWrong Jailbreaking (AIs) Interpretability (ML & AI) Role Science AI Curated 2026 Top Fifty: 14 % 371 A Mechanistic Explanation of Prompt Injection (and why you should study roles) by Charles Ye , Jasmine C. 22nd Jun 2026 19 min read 56 371 Summary We've been building a theory of how prompt injections work under the hood. We show it comes down to how LLMs perceive roles (the humble chat template tags). We use this theory to create new attacks, explain some weird mech interp results, and predict when attacks w

saved by

• Yixiong Hao
• Lydia Nottingham
• Arya Pasumarthi
• Benjamin Grierson
• Lucas Chen
• Goutham N

related reading

• Prompt Injection as Role Confusionrole-confusion.github.io
• Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org
• Refusal in LLMs is mediated by a single direction — LessWronglesswrong.com
• A Three-Layer Model of LLM Psychology — LessWronglesswrong.com
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Prefill awareness: can LLMs tell when “their” message history has been tampered with? — LessWronglesswrong.com
• [2603.12277] Prompt Injection as Role Confusionarxiv.org
• Role-playing vs Self-modelling — LessWronglesswrong.com
• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
• Simulated Users & Sad LLMs1a3orn.com
• Role confusion: sounding like the cause is indistinguishable from being it. — LessWronglesswrong.com