How Data and Verifiers Shape RLVR | Snorkel AI
Learn how RLVR uses verifiable rewards to train models, how training data and verifier design shape performance, and where RLVR still struggles with agents and long-horizon tasks.
Learn how RLVR uses verifiable rewards to train models, how training data and verifier design shape performance, and where RLVR still struggles with agents and long-horizon tasks. TL;DR RLVR trains models using rewards from outcomes that can be checked programmatically, such as correct answers, executable code, or valid structured outputs. Verifiability alone does not produce effective training. Task composition, difficulty, verifier design, and the available optimization budget shape what the model learns. In Snorkel’s experiments, 100 mixed-difficulty examples reached 44.2% mean test…
saved by
related reading
- A Taxonomy of RL Environments for LLM Agentsleehanchung.github.io
- What is RLVR? Reinforcement Learning from Verifiable Rewardsrlvrbook.com
- Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
- Reasoning in General Domains without Verifiersarxiv.org
- Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainsarxiv.org
- Asymmetry of verification and verifier’s rule - Jason Weijasonwei.net
- [2504.13837] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?arxiv.org
- Training a Misaligned Reward Seekeralignment.anthropic.com
- Noisy Data Breaks RLVRddkang.substack.com
- The Verification Horizon: No Silver Bullet for Coding Agent Rewardsarxiv.org
- State of RL for reasoning LLMs | A. Weersaweers.de
- Features as Rewards: Using Interpretability to Reduce Hallucinationsgoodfire.ai