Why Do Naive SFT Filters For Safety Properties Fail? — AI Alignment Forum
This is the fourth in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and ad…
x Why Do Naive SFT Filters For Safety Properties Fail? — AI Alignment Forum AI Frontpage 23 Why Do Naive SFT Filters For Safety Properties Fail? by Josh Engels , Neel Nanda 14th Jun 2026 13 min read 7 23 This is the fourth in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The third post can be found here . Since SFT is the cause for many safety relevant properties , a natural strategy is to filter out rollouts from SFT that have undesirable properties. However, as we show in this section (and in forth
saved by
related reading
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- SFT Drives Gemini’s Safety Properties — LessWronglesswrong.com
- How far does alignment midtraining generalize?alignment.openai.com
- Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
- Teaching Claude Whyalignment.anthropic.com
- SFT Drives Gemini’s Safety Properties — AI Alignment Forumalignmentforum.org
- Josh Engels on X: "New GDM interp research: SFT is a big deal for safety relevant behaviors. We recently investigated root causes for some of Gemini’s behaviors. We were surprised to find that many behaviors actually came from the initial supervised finetuning stage, not later stages like RL! 🧵 https://t.co/mLg87XuXK5" / Xx.com
- [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
- Believe It or Not: How Deeply do LLMs Believe Implanted Facts?alignment.anthropic.com
- SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io
- [2602.05910] Chunky Post-Training: Data Driven Failures of Generalizationarxiv.org
- Refusal in LLMs is mediated by a single direction — LessWronglesswrong.com