Role confusion: sounding like the cause is indistinguishable from being it. — LessWrong
lesswrong.com · 3,861 words · saved by 1 readers
A replication of Prompt Injection as Role Confusion (2026) and why the mechanistic story of prompt injection is harder to pin down than it looks. …
x Role confusion: sounding like the cause is indistinguishable from being it. — LessWrong Interpretability (ML & AI) Jailbreaking (AIs) Role Science AI Frontpage 14 Role confusion: sounding like the cause is indistinguishable from being it. by Owain Mogford 29th Jun 2026 11 min read 9 14 A replication of Prompt Injection as Role Confusion (2026) and why the mechanistic story of prompt injection is harder to pin down than it looks. Epistemic status: I reproduced the direction of the paper's main results on a single consumer GPU (it was faithful in direction but not like for like in magnitude, s
related reading
- A Mechanistic Explanation of Prompt Injection (and why you should study roles) — LessWronglesswrong.com
- Prompt Injection as Role Confusionrole-confusion.github.io
- [2603.12277] Prompt Injection as Role Confusionarxiv.org
- Simulators — LessWronglesswrong.com
- Deep Deceptiveness — LessWronglesswrong.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- the case for CoT unfaithfulness is overstated — LessWronglesswrong.com
- Causal Scrubbing: a method for rigorously testing interpretability hypotheses [Redwood Research] — LessWronglesswrong.com
- Role-playing vs Self-modelling — LessWronglesswrong.com
- How hard is it to inoculate against misalignment generalization? — LessWronglesswrong.com
- Faithful, Interpretable Model Explanations via Causal Abstraction | SAIL Blogai.stanford.edu
- Simulated Users & Sad LLMs1a3orn.com