Policy Gradients Part 1: The REINFORCE Estimator
I have a dirty secret. Well, I actually have many. But one of them is that I never understood the basic algorithms behind reinforcement learning. So I plan to remedy this with a series of blog posts, where I will cover some of the basic RL algorithms, from REINFORCE to …
I have a dirty secret. Well, I actually have many. But one of them is that I never understood the basic algorithms behind reinforcement learning. So I plan to remedy this with a series of blog posts, where I will cover foundational RL algorithms, from REINFORCE to the frontier. Here's the first one, where I cover one of the oldest practical gradient estimators for reinforcement learning, the REINFORCE estimator, and show how it can get away without differentiating through the environment. Setup This is the notation I'll follow throughout the series. An agent interacts with an environment…
saved by
related reading
- Understanding Policy Gradients | John Lambertjohnwlambert.github.io
- Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
- A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
- Keep the gradient flowingfa.bianp.net
- Policy Gradient Algorithms | Lil'Loglilianweng.github.io
- Policy gradient methoden.wikipedia.org
- From REINFORCE to Dr. GRPOlancelqf.github.io
- Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
- RL_Notes__final_.pdfjubayer-ibn-hamid.github.io
- Policy Gradients [Draft]blog.sdan.net
- An Updated Introduction to Reinforcement Learning | Sri's Blogsrianumakonda.com
- Flow Matching Policy Gradientsflowreinforce.github.io