Understanding Policy Gradients | John Lambert
johnwlambert.github.io · 4,040 words · saved by 1 readers
A simple, whitespace theme for academics. Based on [*folio](https://github.com/bogoli/-folio) design.
Table of Contents: Policy Gradients Geometric Intuition Math Background Policy Gradient Theorem The Probability of a Trajectory, Given a Policy The REINFORCE Algorithm Baseline Subtraction Vanilla PG: Python Implementation TD Error as Advantage Function Trust Region Policy Optimization (TRPO) Truncated Natural Gradient Policy Algorithm PPO Policy gradients is a reinforcement learning method, where an agent interacts with the world, taking decisions. After taking an action, the world (i.e. environment) changes, and this process repeats over and over and over. We’ll examine the…
saved by
related reading
- RL_Notes__final_.pdfjubayer-ibn-hamid.github.io
- High variance in RL (and how to measure it)suzyahyah.github.io
- PPO for LLMs: A Guide for Normal Peoplecameronrwolfe.substack.com
- Policy Gradient Algorithms | Lil'Loglilianweng.github.io
- rltheorybook_ABJKS.pdfrltheorybook.github.io
- [1707.06347] Proximal Policy Optimization Algorithmsarxiv.org
- RLAlgsInMDPs.pdfsites.ualberta.ca
- RLHF | John Lambertjohnwlambert.github.io
- Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
- GRPO++: Tricks for Making RL Actually Workcameronrwolfe.substack.com
- The 37 Implementation Details of Proximal Policy Optimization · The ICLR Blog Trackiclr-blog-track.github.io
- A Gallery of Methods Beyond RL — Part I: Sampling Methodsshengyu-feng.github.io