The Art of Scaling Reinforcement Learning for LLMs | Sean Bell, Resolve AI Labs - YouTube

Follow along using the transcript. Follow along using the transcript.

Sean Bell, RL Research Lead at ResolveAI Labs, breaks down the reinforcement learning paper "The Art of Scaling Reinforcement Learning Compute for LLMs" at the SF Systems Reading Group, presented by Greylock Partners. 0:00 Intro and who Sean Bell is 3:00 ResolveAI and AI agents for production incidents 6:50 Live demo of ResolveAI in production 12:10 Why custom models beat off-the-shelf 14:10 The rise of RL and reasoning models 19:00 RL from first principles, SFT and GRPO 41:10 The Art of Scaling RL paper and scaling laws 49:20 Loss functions, CISPO and the train/inference mismatch 1:03:00…

saved by

• Falah Rajput

related reading

• Lil'Loglilianweng.github.io
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• RL Scaling Laws for LLMs - by Cameron R. Wolfe, Ph.D.cameronrwolfe.substack.com
• How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
• State of RL for reasoning LLMs | A. Weersaweers.de
• Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
• Deep Dive into LLMs like ChatGPTyoutube.com
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
• The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
• RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com