[1602.01783] Asynchronous Methods for Deep Reinforcement Learning

We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.

Asynchronous Methods for Deep Reinforcement Learning Volodymyr Mnih1 VMNIH @ GOOGLE . COM Adrià Puigdomènech Badia1 ADRIAP @ GOOGLE . COM Mehdi Mirza1,2 MIRZAMOM @ IRO . UMONTREAL . CA Alex Graves1…

related reading

• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Key Papers in Deep RL - Spinning Up documentationspinningup.openai.com
• [2602.17616] Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMsarxiv.org
• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• [1709.06560] Deep Reinforcement Learning that Mattersarxiv.org
• Deep Q-Networks Explained — LessWronglesswrong.com
• Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningarxiv.org
• Deep RL Bootcamp - Lecturessites.google.com
• NeurIPS-2021-understanding-end-to-end-model-based-reinforcement-learning-methods-as-implicit-parameterization-Supplemental.pdflis.csail.mit.edu
• The idea behind Actor-Critics and how A2C and A3C improve them | AI Summertheaisummer.com