Reinforcement learning is an infrastructure problem

What we've seen helping teams run Reinforcement Learning at scale on Modal. Plus an open-source library to skip the scaffolding.

All posts Back Engineering June 1, 2026 • 10 minute read Reinforcement learning is an infrastructure problem Joy Liu @qjoyliu Member of Technical Staff Charles Frye @charles_irl Member of Technical Staff Peyton Walters @peywalt Member of Technical Staff Reinforcement Learning (RL) to post-train LLMs has exploded in popularity on Modal. We've helped teams of all sizes, from research labs to established enterprises, build training systems to achieve frontier cost-performance from foundation models. What we realized is that the present bottleneck of RL is infrastructure. Today, we want to share w

saved by

• Devin Plumb

related reading

• Composer2.pdfcursor.com
• How To Scale Your Modeljax-ml.github.io
• Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
• Scaling Reinforcement Learning: Environments, Reward Hacking, Agents, Scaling Datasemianalysis.com
• tinker-nomicstinker-nomics.vercel.app
• How We Build Trillion Parameter Reasoning RL with 10% GPUsmacaron.im
• Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructuresemianalysis.com
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
• RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com
• RL Scaling Laws for LLMs - by Cameron R. Wolfe, Ph.D.cameronrwolfe.substack.com
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• Sonya Huang 🐥 on X: "Today's Training Data episode takes us BTS on the infrastructure challenges required to do large RL runs at scale, featuring @ellev3n11 (Composer Lead at @cursor_ai) and @dzhulgakov (Co-Founder at @FireworksAI_HQ). The Cursor team trained Composer 2 on Fireworks by starting with https://t.co/6LLlJlyl8Q" / Xx.com