[2507.06187] The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains

YOU make open access possible! Tell us why you support #openaccess and give to arXiv this week to help keep science open for all. Help | Advanced Search arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status

[2507.06187] The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Artificial Intelligence arXiv:2507.06187 (cs) [Submitted on 8 Jul 2025] Title: The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains Authors: Scott Geng , Hamish Ivison , Chun-Liang Li , Maarten Sap , Jerry Li , Ranjay Krishna , Pang Wei Koh View a PDF of the paper titled The Delta Learning Hypothesis: Prefe

saved by

• Yixiong Hao
• Will Anderson
• Arya Pasumarthi

related reading

• [2507.06187] The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gainsarxiv.org
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• [2509.14786] Pre-training under infinite computearxiv.org
• [2406.11717] Refusal in Language Models Is Mediated by a Single Directionarxiv.org
• [2503.10965] Auditing language models for hidden objectivesarxiv.org
• [2605.12715] Scaling Laws for Mixture Pretraining Under Data Constraintsarxiv.org
• [2512.19428] Attention Is Not What You Needarxiv.org
• arxiv.org/pdf/2505.24832arxiv.org
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• [2309.00267] RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackarxiv.org
• [2508.12773] Online Ensemble Transformer for Accurate Cloud Workload Forecasting in Predictive Auto-Scalingarxiv.org
• [2309.01809] Are Emergent Abilities in Large Language Models just In-Context Learning?arxiv.org