Deep Reinforcement Learning: Pong from Pixels

Musings of a Computer Scientist.

This is a long overdue blog post on Reinforcement Learning (RL). RL is hot! You may have noticed that computers can now automatically learn to play ATARI games (from raw game pixels!), they are beating world champions at Go , simulated quadrupeds are learning to run and leap , and robots are learning how to perform complex manipulation tasks that defy explicit programming. It turns out that all of these advances fall under the umbrella of RL research. I also became interested in RL myself over the last ~year: I worked through Richard Sutton’s book , read through David Silver’s course , watched

related reading

• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Policy Gradient Algorithms | Lil'Loglilianweng.github.io
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• The 37 Implementation Details of Proximal Policy Optimization · The ICLR Blog Trackiclr-blog-track.github.io
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io
• Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
• Debugging Reinforcement Learning Systemsandyljones.com
• Policy Gradient with PyTorchhuggingface.co
• An Updated Introduction to Reinforcement Learning | Sri's Blogsrianumakonda.com
• Deep Q-Networks Explained — LessWronglesswrong.com
• RLHF Bookrlhfbook.com