Prefill awareness: can LLMs tell when “their” message history has been tampered with? — LessWrong

David Africa*, Alex Souly*, Jordan Taylor, Robert Kirk • TLDR: …

x Prefill awareness: can LLMs tell when “their” message history has been tampered with? — LessWrong AI Evaluations AI Personal Blog 86 Prefill awareness: can LLMs tell when “their” message history has been tampered with? by David Africa , alexsouly , Jordan Taylor , RobertKirk 9th Mar 2026 AI Alignment Forum 12 min read 11 86 Ω 33 David Africa*, Alex Souly*, Jordan Taylor, Robert Kirk TLDR: We test whether LLMs can detect when their conversation history has been tampered with (prefill awareness). We find this ability is inconsistent across models and datasets, shallow, and rarely surfaces spon

saved by

• Yixiong Hao
• Kaustubh Kislay
• Harshul Basava

related reading

• Prefill Awareness in Large Language Modelsarxiv.org
• Several frontier models are substantially prefill aware — LessWronglesswrong.com
• Alignment pretraining could backfire — LessWronglesswrong.com
• [2510.04340] Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-timearxiv.org
• You Can Catch Sleeper Agents by Teaching Another Model to Imitate Them — LessWronglesswrong.com
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentalignmentpretraining.ai
• A Mechanistic Explanation of Prompt Injection (and why you should study roles) — LessWronglesswrong.com
• User awareness in frontier modelstransluce.org
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org