Open Problems of Reinforcement Learning

N/A

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Stephen Casper,∗ MIT CSAIL, scasper@mit.edu Xander Davies,∗ Harvard University Claudia Shi, Columbia University Thomas Krendl Gilbert, Cornell Tech Jérémy Scheurer, Apollo Research Javier Rando, ETH Zurich…

related reading

• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• Constitutional AI: Harmlessness from AI Feedbackarxiv.org
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• [2309.00267] RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackarxiv.org
• RLHF | John Lambertjohnwlambert.github.io
• [2402.14740] Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMsarxiv.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• Training language models to follow instructions with human feedback.pdfproceedings.neurips.cc
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org