SFT Drives Gemini’s Safety Properties — LessWrong
lesswrong.com · 807 words · saved by 5 readers
This is the third in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adj…
x SFT Drives Gemini’s Safety Properties — LessWrong AI Frontpage 89 SFT Drives Gemini’s Safety Properties by Josh Engels , Arthur Conmy , bilalchughtai , Neel Nanda 13th Jun 2026 AI Alignment Forum 2 min read 4 89 Ω 33 This is the third in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The second post can be found here . In this short post, we describe a surprising finding: most safety relevant properties in Gemini seem to be caused by the combination of pretraining and SFT, not other training stages
saved by
related reading
- SFT Drives Gemini’s Safety Properties — AI Alignment Forumalignmentforum.org
- Josh Engels on X: "New GDM interp research: SFT is a big deal for safety relevant behaviors. We recently investigated root causes for some of Gemini’s behaviors. We were surprised to find that many behaviors actually came from the initial supervised finetuning stage, not later stages like RL! 🧵 https://t.co/mLg87XuXK5" / Xx.com
- Why Do Naive SFT Filters For Safety Properties Fail? — AI Alignment Forumalignmentforum.org
- A Summary of Recent Work (July 2026)gdmalignment.substack.com
- Why study alignment interventions on pre-RL checkpoints? — LessWronglesswrong.com
- Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivesarxiv.org
- When Should We Introduce Safety Interventions During Pretraining?arxiv.org
- Persona Features Control Emergent Misalignmentarxiv.org
- [2510.27062] Consistency Training Helps Stop Sycophancy and Jailbreaksarxiv.org
- A Pragmatic Vision for Interpretability — LessWronglesswrong.com
- [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
- Gemini 3 is Evaluation-Paranoid and Contaminated — LessWronglesswrong.com