[2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviors

Abstract:When model developers or users fine-tune an LLM, this can induce behaviors that are unexpected, deliberately harmful, or hard to detect. It would be far easier to audit LLMs if they could simply describe their behaviors in natural language. Here, we study a scalable approach to rapidly identify learned behaviors of many LLMs derived from a shared base LLM. Given a model $M$, our method works by finetuning models $M_i$ from $M$ with implanted behaviors $b_i$; the $(M_i, b_i)$ pairs serve as labeled training data. We then train an \emph{introspection adapter} (IA): a single LoRA adapter jointly trained across the finetunes $M_i$ to cause them to verbalize their implanted behaviors. We find that this IA induces self-description of learned behaviors even in finetunes of $M$ that were trained in very different ways from the $M_i$. For example, IAs generalize to AuditBench, achieving state-of-the-art at identifying explicitly hidden concerning behaviors. IAs can also be used to detect encrypted finetuning API attacks. They scale favorably with model size and training data diversity. Overall, our results suggest that IAs are a scalable, effective, and practically useful approach to auditing fine-tuned LLMs.

# link_2fdo21eijlt.pdf ## Metadata - PDFFormatVersion=1.7 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Author=Keshav Shenoy; Li Yang; Abhay Sheshadri; Sören Mindermann; Jack Lindsey; Sam Marks; Rowan Wang - Creator=arXiv GenPDF (tex2pdf:a6404ea) - Custom.DOI=https://doi.org/10.48550/arXiv.2604.16812 - Custom.License=http://creativecommons.org/licenses/by/4.0/ - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.141592653-2.6-1.40.28 (TeX Live 2025) kpathsea version 6.4.1 - Custom.arXivID=https://arxiv.org/abs/2604.1

saved by

• Lydia Nottingham
• Eric Huang
• Emil Ryd
• Neil Rathi

related reading

• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2501.11120] Tell me about yourself: LLMs are aware of their learned behaviorsarxiv.org
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• [2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingarxiv.org
• [2503.10965] Auditing language models for hidden objectivesarxiv.org
• How confessions can keep language models honest | OpenAIopenai.com
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• LLMs can learn about themselves by introspection — LessWronglesswrong.com
• Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Traininggoodfire.ai
• [2410.13787] Looking Inward: Language Models Can Learn About Themselves by Introspectionarxiv.org
• [2410.13787] Looking Inward: Language Models Can Learn About Themselves by Introspectionarxiv.org
• You Can Catch Sleeper Agents by Teaching Another Model to Imitate Them — LessWronglesswrong.com