[2506.10947] Spurious Rewards: Rethinking Training Signals in RLVR
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack
[2506.10947] Spurious Rewards: Rethinking Training Signals in RLVR Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Artificial Intelligence arXiv:2506.10947 (cs) [Submitted on 12 Jun 2025 ( v1 ), last revised 25 Feb 2026 (this version, v2)] Title: Spurious Rewards: Rethinking Training Signals in RLVR Authors: Rulin Shao , Shuyue Stella Li , Rui Xin , Scott Geng , Yiping Wang , Sewoong Oh , Simon Shaolei Du , Nathan Lambert , Sewon Min , Ranjay Krishna , Yulia Tsvetkov , Hannaneh Haj
related reading
- Reinforcement Learning With Verifiable Rewards: How Data and Verifiers Shape RLVRsnorkel.ai
- State of RL for reasoning LLMs | A. Weersaweers.de
- Reasoning in General Domains without Verifiersarxiv.org
- Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
- DeepSeek-R1arxiv.org
- What is RLVR? Reinforcement Learning from Verifiable Rewardsrlvrbook.com
- [2504.13837] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?arxiv.org
- The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
- Noisy Data Breaks RLVRddkang.substack.com
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- The Invisible Leash: Why RLVR May Not Escape Its Originarxiv.org
- You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectoriesarxiv.org