Optimisation over non-stationary distributions creates weirder minds — LessWrong
TLDR: Sequentially mixing training objectives incentivises different training dynamics depending on the distinguishability of the training environmen…
x Optimisation over non-stationary distributions creates weirder minds — LessWrong AI Frontpage 36 Optimisation over non-stationary distributions creates weirder minds by Samuel Ratnam , Pjain 6th Jun 2026 4 min read 8 36 TLDR: Sequentially mixing training objectives incentivises different training dynamics depending on the distinguishability of the training environments and the amount of pressure for shared circuitry. We classify these patterns into three classes: ecological generalists, conditional policies, and strategy churn. We suggest that careful consideration of the pressures of non-st
Explore this link on the map →related reading
- On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
- SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io
- How do LLMs generalize when we do training that is intuitively compatible with two off-distribution behaviors? — LessWronglesswrong.com
- Composer2.pdfcursor.com
- llm assistant personas seem increasingly incoherent (some subjective observations) — LessWronglesswrong.com
- Just Ask for Generalization | Eric Jangevjang.com
- State of RL for reasoning LLMs | A. Weersaweers.de
- Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
- Models Don't "Get Reward" — LessWronglesswrong.com
- Training a Reward Hacker Despite Perfect Labels — LessWronglesswrong.com
- Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io
- What I learned this week - Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RLdwarkesh.com