Reinforcement learning is an infrastructure problem

What we've seen helping teams run Reinforcement Learning at scale on Modal. Plus an open-source library to skip the scaffolding.

All posts Back Engineering June 1, 2026 • 10 minute read Reinforcement learning is an infrastructure problem Joy Liu @qjoyliu Member of Technical Staff Charles Frye @charles_irl Member of Technical Staff Peyton Walters @peywalt Member of Technical Staff Reinforcement Learning (RL) to post-train LLMs has exploded in popularity on Modal. We've helped teams of all sizes, from research labs to established enterprises, build training systems to achieve frontier cost-performance from foundation models. What we realized is that the present bottleneck of RL is infrastructure. Today, we want to share w

saved by

• Devin Plumb

related reading

• Composer2.pdfcursor.com
• Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
• Low Latency and Model Training at Modalrhea24.github.io
• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
• The upcoming GPT-3 moment for RL | Mechanize, Inc.mechanize.work
• The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
• RL Post-Training on Macs | Pluralis Researchpluralis.ai
• RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com
• Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructuresemianalysis.com
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
• tinker-nomicstinker-nomics.vercel.app
• Scaling Reinforcement Learning: Environments, Reward Hacking, Agents, Scaling Datasemianalysis.com