The idea behind Actor-Critics and how A2C and A3C improve them | AI Summer

Actor critics, A2C, A3C

It’s time for some Reinforcement Learning. This time our main topic is Actor-Critic algorithms, which are the base behind almost every modern RL method from Proximal Policy Optimization to A3C. So, to understand all those new techniques, you should have a good grasp of what Actor-Critic are and how they work. But don’t be in a hurry. Let’s refresh for a moment on our previous knowledge. As you may know, there are two main types of RL methods out there: Value Based: They try to find or approximate the optimal value function, which is a mapping between an action and a value. The higher the value

related reading

• Advantage Actor Critic (A2C)huggingface.co
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• RL ALGOk-a.in
• Policy Gradient Algorithms | Lil'Loglilianweng.github.io
• Soft Actor-Critic - Spinning Up documentationspinningup.openai.com
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Actor-Critic Algorithmspapers.nips.cc
• Extended Data Fig. 5: Training progress. | Naturenature.com
• The 37 Implementation Details of Proximal Policy Optimization · The ICLR Blog Trackiclr-blog-track.github.io
• Part 2: Kinds of RL Algorithms - Spinning Up documentationspinningup.openai.com
• [1602.01783] Asynchronous Methods for Deep Reinforcement Learningarxiv.org
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io