Orbit - Ultra-efficient RL Pipeline

Orbit — Ultra-efficient RL Pipeline

Orbit is an open-source RL framework for memory-efficient post-training of trillion-parameter LLMs. By minimizing the train-rollout gap in RL, Orbit is able to effortlessly train Kimi-K2.6 and DeepSeek V4-pro at a single-node setting . Five rollout architectures, one shared clock, clear win for Orbit's design Watch the bottleneck move. We start fully serial with a full-weight push (basesline 1 below), then test the tokens "is the heavy push the bottleneck?" . We gradually turn the baseline into Orbit's final design (+ overlap (async); + adapter-native push; + a non-blocking double-buffered swa

saved by

• Alex K

related reading

• You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectoriesarxiv.org
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusionarxiv.org
• LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
• How We Build Trillion Parameter Reasoning RL with 10% GPUsmacaron.im
• Composer2.pdfcursor.com
• RL at 1T Scale: prime-rl Performance Deep Diveprimeintellect.ai
• State of RL for reasoning LLMs | A. Weersaweers.de
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelsarxiv.org
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
• RL Scaling Laws for LLMs - by Cameron R. Wolfe, Ph.D.cameronrwolfe.substack.com
• LLM Resourcesforrestbicker.com