Bitter Lessons from Distillation Robustifies Unlearning

Bruce W. Lee. Machine Unlearning. Distillation. AI Safety.

Introduction My collaborators and I wrote a paper titled "Distillation Robustifies Unlearning" a few months ago. I'm writing this post to communicate what I think our paper actually says about the problem of unlearning. This is more of a personal account than a group statement. This post is also somewhat intuition-heavy because the goal is to describe the worldview that emerged from working on a paper. Additionally, I'll argue that distillation is an excellent opportunity for a safety intervention that also happens to align with economic incentives. Therefore, I think any future effort to unde

saved by

• Lydia Nottingham
• Jo J.
• Neil Rathi
• Christine Ye

related reading

• Distillation Robustifies Unlearning — LessWronglesswrong.com
• On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Dataalignment.anthropic.com
• AI safety techniques leveraging distillation — LessWronglesswrong.com
• Large Language Models Must Be Taught to Know What They Don't Knowarxiv.org
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• The distillation double bind: Distilling misaligned models either transfers misalignment or it doesn'tblog.redwoodresearch.org
• Incriminating misaligned AI models via distillation — LessWronglesswrong.com
• Distillation Walkthroughvladfeinberg.com
• What I learned this week - Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RLdwarkesh.com
• SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io