Incriminating misaligned AI models via distillation — LessWrong

Suppose we have a dangerous misaligned AI that can fool alignment audits, and distill it into a student model. Two things can happen: …

x Incriminating misaligned AI models via distillation — LessWrong AI Auditing Redwood Research AI Frontpage 2026 Top Fifty: 14 % 117 Incriminating misaligned AI models via distillation by Alek Westover , SebastianP , Alex Mallen , Jozdien , Alexa Pan , Julian Stastny , Vivek Hebbar 15th May 2026 6 min read 12 117 Suppose we have a dangerous misaligned AI that can fool alignment audits, and distill it into a student model. Two things can happen: Misalignment fails to transfer to the student. If so, we get a fairly capable benign model. Misalignment transfers to the student. The student might al

saved by

• Brady Gho
• Seth Lifland

related reading

• The distillation double bind: Distilling misaligned models either transfers misalignment or it doesn'tblog.redwoodresearch.org
• Automated Alignment is Harder Than You Think — LessWronglesswrong.com
• Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWronglesswrong.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• AI safety techniques leveraging distillation — LessWronglesswrong.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Deep Deceptiveness — LessWronglesswrong.com
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Auditing language models for hidden objectives — LessWronglesswrong.com
• Alignment Faking Mitigationsalignment.anthropic.com
• Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Dataalignment.anthropic.com