SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning | alphaXiv

View recent discussion. Abstract: Vision-Language-Action (VLA) models have recently emerged as a powerful paradigm for robotic manipulation. Despite substantial progress enabled by large-scale pretraining and supervised fine-tuning (SFT), these models face two fundamental challenges: (i) the scarcity and high cost of large-scale human-operated robotic trajectories required for SFT scaling, and (ii) limited generalization to tasks involving distribution shift. Recent breakthroughs in Large Reasoning Models (LRMs) demonstrate that reinforcement learning (RL) can dramatically enhance step-by-step reasoning capabilities, raising a natural question: Can RL similarly improve the long-horizon step-by-step action planning of VLA? In this work, we introduce SimpleVLA-RL, an efficient RL framework tailored for VLA models. Building upon veRL, we introduce VLA-specific trajectory sampling, scalable parallelization, multi-environment rendering, and optimized loss computation. When applied to OpenVLA-OFT, SimpleVLA-RL achieves SoTA performance on LIBERO and even outperforms $\pi_0$ on RoboTwin 1.0\&2.0 with the exploration-enhancing strategies we introduce. SimpleVLA-RL not only reduces dependence on large-scale data and enables robust generalization, but also remarkably surpasses SFT in real-world tasks. Moreover, we identify a novel phenomenon ``pushcut'' during RL training, wherein the policy discovers previously unseen patterns beyond those seen in the previous training process. Github: this https URL

Submitted 11 Sept 2025 Abstract Vision-Language-Action (VLA) models have recently emerged as a powerful paradigm for robotic manipulation. Despite substantial progress enabled by large-scale pretraining and supervised fine-tuning (SFT), these models face two fundamental challenges: (i) the scarcity and high cost of large-scale human-operated robotic trajectories required for SFT scaling, and (ii) limited generalization to tasks involving distribution shift. Recent breakthroughs in Large Reasoning Models (LRMs) demonstrate that reinforcement learning (RL) can dramatically enhance…

saved by

• arya

related reading

• Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Trainingalphaxiv.org
• $π_0$: A Vision-Language-Action Flow Model for General Robot Controlalphaxiv.org
• State of Vision-Language-Action (VLA) Research at ICLR 2026 – Moritz Reussmbreuss.github.io
• Emergence of Human to Robot Transfer in Vision-Language-Action Modelspi.website
• Precise Manipulation with Efficient Online RLpi.website
• How Claude Performs on Robotics Tasks \ Anthropicanthropic.com
• A VLA with Open-World Generalizationpi.website
• RT-2: Vision-Language-Action Modelsrobotics-transformer2.github.io
• pistar06.pdfpi.website
• Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blogdeveloper.nvidia.com
• Vision-Language-Action (VLA) Models: A Review of Recent Progressxxxxyu.github.io
• A VLA that Learns from Experiencepi.website