ash80/RLHF_in_notebooks: RLHF (Supervised fine-tuning, reward model, and PPO) step-by-step in 3 Jupyter notebooks ·

RLHF (Supervised fine-tuning, reward model, and PPO) step-by-step in 3 Jupyter notebooks

Reinforcement Learning from Human Feedback (RLHF) in Notebooks This repository provides a reference implementation for Reinforcement Learning from Human Feedback (RLHF) [Paper] framework presented in the RLHF from scratch, step-by-step, in code YouTube video. Overview of RLHF RLHF is a method for aligning large language models (LLMs), like GPT-3 or GPT-2, to better meet users' intents. It is essentially a reinforcement learning approach, where rather than directly getting the reward or feedback from some environemnt or human, it instead trains a reward model that learns to mimic that…

saved by

• Katherine Driscoll

related reading

• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• RLHF & Post-Training Course by Nathan Lambertrlhfbook.com
• RLHF | John Lambertjohnwlambert.github.io
• Introduction | RLHF and Post-Training Book by Nathan Lambertrlhfbook.com
• Fine-tune Llama 2 with DPOhuggingface.co
• rlhfbook.com/book.pdfrlhfbook.com
• The N Implementation Details of RLHF with PPO | ICLR Blogposts 2024iclr-blogposts.github.io
• LLM Training: RLHF and Its Alternativesmagazine.sebastianraschka.com
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io
• [2402.14740] Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMsarxiv.org
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• Reinforcement learning from human feedback - Wikipediaen.wikipedia.org