flâneur

Josh Engels on X: "New GDM interp research: SFT is a big deal for safety relevant behaviors. We recently investigated root causes for some of Gemini’s behaviors. We were surprised to find that many behaviors actually came from the initial supervised finetuning stage, not later stages like RL! 🧵 https://t.co/mLg87XuXK5" / X

x.com · 208 words · saved by 1 readers

New GDM interp research: SFT is a big deal for safety relevant behaviors. We recently investigated root causes for some of Gemini’s behaviors. We were surprised to find that many behaviors actually came from the initial supervised finetuning stage, not later stages like RL! 🧵 https://t.co/mLg87XuXK5

@JoshAEngels: New GDM interp research: SFT is a big deal for safety relevant behaviors. We recently investigated root causes for some of Gemini’s behaviors. We were surprised to find that many behaviors actually came from the initial supervised finetuning stage, not later stages like RL! @JoshAEngels: This work is part of a series of research updates from our team: @JoshAEngels: My intuition for what’s going on: when we distill from a set of rollouts in SFT, we are establishing a strong prior on a certain assistant persona. Traits of that persona mostly aren’t explicitly optimized against i

saved by

related reading