Reinforcement learning from human feedback - Wikipedia

In machine learning, reinforcement learning from human feedback (RLHF), also known as reinforcement learning from human preferences, is a technique to align an intelligent agent to human preferences. In classical reinforcement learning, the goal of such an agent is to learn a function that guides its behavior called a policy. This function learns to maximize the reward it receives from a separate reward function based on its task performance.[1] In the case of human preferences, however, it tends to be difficult to define explicitly a reward function that approximates human preferences. Therefore, RLHF seeks to train a "reward model" directly from human feedback.[2] The reward model is first trained in a supervised fashion—independently from the policy being optimized—to predict if a response to a given prompt is good (high reward) or bad (low reward) based on ranking data collected from human annotators. This model is then used as a reward function to improve an agent's policy through

Reinforcement learning from human feedback - Wikipedia Jump to content From Wikipedia, the free encyclopedia Machine learning technique High-level overview of reinforcement learning from human feedback Part of a series on Machine learning and data mining Paradigms Supervised learning Unsupervised learning Semi-supervised learning Self-supervised learning Reinforcement learning Meta-learning Online learning Batch learning Curriculum learning Rule-based learning Neuro-symbolic AI Neuromorphic engineering Quantum machine learning Problems Classification Generative modeling Regression Clustering D

related reading

• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• RLHF | John Lambertjohnwlambert.github.io
• A simple technical explanation of RLH(AI)F | Kairos.fmkairos.fm
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Deep Reinforcement Learning from Human Preferencesproceedings.neurips.cc
• RLHF Bookrlhfbook.com
• RLHF Bookrlhfbook.com
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• Reinforcement learning - Wikipediaen.wikipedia.org
• Nash Learning from Human Feedbackarxiv.org
• GitHub - ash80/RLHF_in_notebooks: RLHF (Supervised fine-tuning, reward model, and PPO) step-by-step in 3 Jupyter notebooksgithub.com