Model Organisms for Emergent Misalignment — LessWrong

We show emergent misalignment is a robust and safety-relevant result, and open-source improved model organisms to accelerate future work.

x Model Organisms for Emergent Misalignment — LessWrong Emergent Misalignment MATS Program Interpretability (ML & AI) AI Frontpage 2025 Top Fifty: 15 % 120 Model Organisms for Emergent Misalignment by Anna Soligo , Edward Turner , Mia Taylor , Senthooran Rajamanoharan , Neel Nanda 16th Jun 2025 AI Alignment Forum 7 min read 19 120 Ω 56 Ed and Anna are co-first authors on this work. TL;DR Emergent Misalignment (EM) showed that fine-tuning LLMs on insecure code caused them to become broadly misaligned. We show this is a robust and safety-relevant result, and open-source improved model organisms

saved by

• Vyom Pathak

related reading

• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Model Organisms for Emergent Misalignmentarxiv.org
• Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs 49 This paper contains model-generated content that might be offensive. 49arxiv.org
• [2506.11613] Model Organisms for Emergent Misalignmentarxiv.org
• Open problems in emergent misalignment — LessWronglesswrong.com
• Chapter 4: Alignment Science - ARENAlearn.arena.education
• We need a better way to evaluate emergent misalignment — LessWronglesswrong.com
• Narrow Misalignment is Hard, Emergent Misalignment is Easy — LessWronglesswrong.com
• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Chapter 4: Alignment Science - ARENAlearn.arena.education
• Teaching Claude why \ Anthropicanthropic.com