RL at 1T Scale: prime-rl Performance Deep Dive
prime-rl 0.6.0 trains trillion-parameter MoE models on heavy agentic workloads at the highest efficiency. A deep dive into the inference and training optimizations behind it — from FP8 and wide expert parallelism to P/D disaggregation, router replay, and 3-D parallelism.
RL at 1T Scale: prime-rl Performance Deep Dive Today we are releasing prime-rl version 0.6.0. This version enables us (and you) to train models of trillion-parameter scale on heavy agentic workloads at the highest efficiency. We have been relentlessly optimizing our RL infrastructure to maximize performance on large MoE models, reducing the cost, time and suffering required to post-train OSS models on agentic workflows. We are able to train GLM-5 on SWE tasks at up to 131k sequence length, with sub-5-minute step times and a batch size of 256 rollouts, on only 28 H200 nodes. In this blog we wil
saved by
related reading
- GLM-5.2 RL weight transfer in 4 seconds using NIXL and ModelExpressprimeintellect.ai
- Journey to 2-second Inter-node RL Weight Transferle.qun.ch
- Updating 1T parameters in seconds — P2P weight transfer in Large Scale Distributed RL - LMSYS Orglmsys.org
- feat: algorithm abstraction — named algorithm classes + inline frozen-model references (grpo, opd, sft_distill, self_distill, echo) by hallerite · Pull Request #2746 · PrimeIntellect-ai/prime-rl · GitHubgithub.com
- Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
- Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
- The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- How We Build Trillion Parameter Reasoning RL with 10% GPUsmacaron.im
- Orbit - Ultra-efficient RL Pipelinespherelab.ai
- State of RL for reasoning LLMs | A. Weersaweers.de