Trajectory Improvement and Reward Learning from Comparative Language Feedback

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions.

Trajectory Improvement and Reward Learning from Comparative Language Feedback Zhaojing Yang University of Southern California Miru Jun University of Southern California Jeremy Tien University of California, Berkeley Stuart J. Russell University of California, Berkeley Anca Dragan University of California, Berkeley Erdem Bıyık University of Southern California Abstract Learning from human feedback has gained traction in fields like robotics and natural language processing in recent years. While prior works mostly rely on human feedback in the form of comparisons, language is a preferable modali

related reading

• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• Deep Reinforcement Learning from Human Preferencesproceedings.neurips.cc
• [2109.01115] Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationarxiv.org
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org
• Training language models to follow instructions with human feedback.pdfproceedings.neurips.cc
• 2401.10020.pdfarxiv.org
• [2309.00267] RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackarxiv.org
• Reward Isn’t Free: Supervising Robot Learning with Language and Video from the Web | SAIL Blogai.stanford.edu
• [2204.05186] Correcting Robot Plans with Natural Language Feedbackar5iv.labs.arxiv.org
• RLHF | John Lambertjohnwlambert.github.io
• Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts | RLHFlowrlhflow.github.io