Reformist Reinforcement Learning - by Ben Recht - arg min

The final week of the semester coincides with the annual AI Winter megaconference, and so I’ll lean into posting about AI this week. And given the spiritual syzygy, I’m going to focus on a fascinating intersection between AI methods and AI culture: Reinforcement learning. Reinforcement learning is a weird subfield of AI that has plagued me since I got to Berkeley. I have written about it countless times. I have an old, pre-substack blog series about it. I wrote a whole survey about it. I wrote more recent substack posts about it. I helped found a conference about it! But even after all of that, I still have a tough time explaining what reinforcement learning is. It’s incredibly hard to pin down. That’s because it’s more of a culture than a body of technical results. If you read the main book by the Turing Award winners in the field, it’s supposedly the foundation of all automated decision-making, mimicking natural intelligence in people and animals.1 If you look at what the heuristics

Reformist Reinforcement Learning What if we just begin and end with policy gradient? Ben Recht Dec 01, 2025 51 25 4 Share The final week of the semester coincides with the annual AI Winter megaconference, and so I’ll lean into posting about AI this week. And given the spiritual syzygy, I’m going to focus on a fascinating intersection between AI methods and AI culture : Reinforcement learning. Reinforcement learning is a weird subfield of AI that has plagued me since I got to Berkeley. I have written about it countless times. I have an old, pre-substack blog series about it. I wrote a whole sur

saved by

• Michael Li

related reading

• Reward is not the optimization target — LessWronglesswrong.com
• RLHF & Post-Training Course by Nathan Lambertrlhfbook.com
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• Reinforcement Learning: Machine Learning Meets Control Theoryyoutube.com
• Debugging Reinforcement Learning Systemsandyljones.com
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• ML Mentorship: Some Q/A about RLevjang.com
• [2510.13651] What is the objective of reasoning with reinforcement learning?arxiv.org
• You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
• SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io