Pedagogical RL: Teaching Models to Teach Themselves from Privileged Information - Noah Ziems

Souradip Chakraborty*,1,2, Noah Ziems*,1,3, Furong Huang2, Meng Jiang3, Amrit Singh Bedi4, Omar Khattab1 1MIT   2UMD   3UND   4UCF *Equal contribution

Souradip Chakraborty *,1,2 , Noah Ziems *,1,3 , Furong Huang 2 , Meng Jiang 3 , Amrit Singh Bedi 4 , Omar Khattab 1 1 MIT 2 UMD 3 ND 4 UCF * Equal contribution May 14, 2026 · Announcement tweet Typical reinforcement learning and on-policy distillation algorithms rely on privileged information like labeled final answers or execution feedback to evaluate rollouts , but do not actually benefit from them for finding good rollouts . If your model can’t already stumble upon successful trajectories, RL simply stalls. In this post, we ask: Can we leverage privileged information to actively samp

saved by

• Uzay Girit
• Timothy Kostolansky
• Ishan Mukherjee
• Christina Lee

related reading

• On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
• [2606.10346] Reasoning or Memorization? Direction-Aware Diversity Exploration in LLM Reinforcement Learningarxiv.org
• Reinforcement Learning for Knowledge Awareness – kalomaze's kalomazing blogkalomaze.bearblog.dev
• State of RL for reasoning LLMs | A. Weersaweers.de
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• [2601.18734] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelsarxiv.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• On-Policy Distillation: Promise, Pitfalls, and Prospectslouieworth.github.io
• [2604.00626] A Survey of On-Policy Distillation for Large Language Modelsarxiv.org
• [2602.19362] LLMs Can Learn to Reason Via Off-Policy RLarxiv.org
• [2103.06326] S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningarxiv.org
• Speeding up RL with high-leverage samples | Applied Computeappliedcompute.com