RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold

N/A

RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold 1 1 2 3 1 2 Amrith Setlur , Saurabh Garg , Xinyang (Young) Geng , Naman Garg , Virginia Smith and Aviral Kumar 1 2 3 Carnegie Mellon…

related reading

• On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
• [2409.19759] Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMsarxiv.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• State of RL for reasoning LLMs | A. Weersaweers.de
• Optimizing LLM Test-Time Compute Involves Solving a Meta-RL Problem – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
• [2504.13837] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?arxiv.org
• LLM Reasoning via One Examplearxiv.org
• Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelsarxiv.org
• Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelsarxiv.org