Understanding Policy Gradients | John Lambert
johnwlambert.github.io · 4,040 words · saved by 1 readers
A simple, whitespace theme for academics. Based on [*folio](https://github.com/bogoli/-folio) design.
Table of Contents: Policy Gradients Geometric Intuition Math Background Policy Gradient Theorem The Probability of a Trajectory, Given a Policy The REINFORCE Algorithm Baseline Subtraction Vanilla PG: Python Implementation TD Error as Advantage Function Trust Region Policy Optimization (TRPO) Truncated Natural Gradient Policy Algorithm PPO Policy gradients is a reinforcement learning method, where an agent interacts with the world, taking decisions. After taking an action, the world (i.e. environment) changes, and this process repeats over and over and over. We’ll examine the…
saved by
related reading
- RL_Notes__final_.pdfjubayer-ibn-hamid.github.io
- Policy Gradient Algorithms | Lil'Loglilianweng.github.io
- PPO for LLMs: A Guide for Normal Peoplecameronrwolfe.substack.com
- High variance in RL (and how to measure it)suzyahyah.github.io
- [1707.06347] Proximal Policy Optimization Algorithmsarxiv.org
- rltheorybook_ABJKS.pdfrltheorybook.github.io
- GRPO++: Tricks for Making RL Actually Workcameronrwolfe.substack.com
- Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
- Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
- [2602.19362] LLMs Can Learn to Reason Via Off-Policy RLarxiv.org
- Policy gradient methoden.wikipedia.org
- State of RL for reasoning LLMs | A. Weersaweers.de