Reinforcement learning is an infrastructure problem

What we've seen helping teams run Reinforcement Learning at scale on Modal. Plus an open-source library to skip the scaffolding.

All posts Back Engineering June 1, 2026 • 10 minute read Reinforcement learning is an infrastructure problem Joy Liu @qjoyliu Member of Technical Staff Charles Frye @charles_irl Member of Technical Staff Peyton Walters @peywalt Member of Technical Staff Reinforcement Learning (RL) to post-train LLMs has exploded in popularity on Modal. We've helped teams of all sizes, from research labs to established enterprises, build training systems to achieve frontier cost-performance from foundation models. What we realized is that the present bottleneck of RL is infrastructure. Today, we want to share w

saved by

• Devin Plumb

related reading

• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
• Low Latency and Model Training at Modalrhea24.github.io
• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io
• Debugging Reinforcement Learning Systemsandyljones.com
• Modal: High-performance AI infrastructuremodal.com
• Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
• Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructuresemianalysis.com
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
• Training Imperativesdan.io
• Rethinking RL Infra for Agents | B'Logbillxbf.github.io
• RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com