ash80/RLHF_in_notebooks: RLHF (Supervised fine-tuning, reward model, and PPO) step-by-step in 3 Jupyter notebooks ·

RLHF (Supervised fine-tuning, reward model, and PPO) step-by-step in 3 Jupyter notebooks

Reinforcement Learning from Human Feedback (RLHF) in Notebooks This repository provides a reference implementation for Reinforcement Learning from Human Feedback (RLHF) [Paper] framework presented in the RLHF from scratch, step-by-step, in code YouTube video. Overview of RLHF RLHF is a method for aligning large language models (LLMs), like GPT-3 or GPT-2, to better meet users' intents. It is essentially a reinforcement learning approach, where rather than directly getting the reward or feedback from some environemnt or human, it instead trains a reward model that learns to mimic that…

saved by

• Katherine Driscoll

related reading

• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• RLHF | John Lambertjohnwlambert.github.io
• GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
• A simple technical explanation of RLH(AI)F | Kairos.fmkairos.fm
• LLM Training: RLHF and Its Alternativesmagazine.sebastianraschka.com
• RLHF Bookrlhfbook.com
• The N Implementation Details of RLHF with PPO | ICLR Blogposts 2024iclr-blogposts.github.io
• Introduction | RLHF and Post-Training Book by Nathan Lambertrlhfbook.com
• RLHF & Post-Training Course by Nathan Lambertrlhfbook.com
• Reinforcement learning from human feedback - Wikipediaen.wikipedia.org
• Fine-tune Llama 2 with DPOhuggingface.co