ar0cket1 on X: "Solving OPSD (basically)" / X

https://t.co/QL4SkUecDr

@ar0cket1: Solving OPSD (basically) self hinted teachers will likely be common practice in a few months for RL, here is the progress and findings I have made: *note: this is a continuation of my initial blog ar0cket1 @ar0cket1 · May 12 Article On Policy Self Distillation I’ve been working on solving OPSD for a week now (and a little more). The goal is RL like upper bound, with OPD like sample efficiency while being stable. This is the results I’ve gotten so far +... 6 9 142 53K Context Just to contextualize everything: OPSD’s goal can be thought of as achieving OPD like sample efficiency w

saved by

• Tom Zheng
• Bhagyesh Kumar

related reading

• SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io
• On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
• On-Policy Distillation: Promise, Pitfalls, and Prospectslouieworth.github.io
• [2601.18734] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelsarxiv.org
• Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelsarxiv.org
• [2604.13010] Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationarxiv.org
• Dense, on-policy, or both?baseten.co
• Pedagogical RL: Teaching Models to Teach Themselves from Privileged Information - Noah Ziemsnoahziems.com
• On the Geometry of On-Policy Distillationarxiv.org
• Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Trainingarxiv.org
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io