Deep Q-Networks Explained — LessWrong

Note: This is a long post. The post is structured in such a way that not everyone needs to read everything - Sections 1 and 2 are skippable background information, and Sections 4 and 5 go into technical detail that not everybody wants or needs to know. Section 3 on its own is sufficient to gain a high-level understanding of DQN if you already know how reinforcement learning works. This post was made as my final project for the AGI Safety Fundamentals course. This post aims to provide a distillation of Deep Q-Networks, neural networks trained via Deep Q-Learning. The algorithm, usually just referred to as DQN, is the algorithm that first put deep reinforcement learning on the map. The 2013 paper is also the first paper in the Key Papers section of the excellent Spinning Up In Deep RL resource. As a result, anyone who wants to understand reinforcement learning is likely to start here. Since reading and replicating papers can be difficult, this resource aims to make the learning curve a

x Deep Q-Networks Explained — LessWrong Distillation & Pedagogy Machine Learning (ML) Reinforcement learning AI Frontpage 58 Deep Q-Networks Explained by Jay Bailey 13th Sep 2022 24 min read 8 58 Note: This is a long post. The post is structured in such a way that not everyone needs to read everything - Sections 1 and 2 are skippable background information, and Sections 4 and 5 go into technical detail that not everybody wants or needs to know. Section 3 on its own is sufficient to gain a high-level understanding of DQN if you already know how reinforcement learning works. This post was made a

saved by

• Claire Wang
• Lydia Nottingham
• Yudhister Joel Kumar

related reading

• Reinforcement Learning in Newcomblike Problemsproceedings.neurips.cc
• Key Papers in Deep RL - Spinning Up documentationspinningup.openai.com
• Debugging Reinforcement Learning Systemsandyljones.com
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Learn Reinforcement Learning (2) - DQN · greentec's bloggreentec.github.io
• The Inverted Pendulum Problem with Deep Reinforcement Learning | by Saif Uddin Mahmud | Dabbler in Destress | Mediummedium.com
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• An Updated Introduction to Reinforcement Learning | Sri's Blogsrianumakonda.com
• Q-learning - Wikipediaen.wikipedia.org
• Q-learning is not yet scalableseohong.me
• Neural networks and deep learningneuralnetworksanddeeplearning.com