The Promise of Hierarchical Reinforcement Learning

This idea of temporal abstraction, once incorporated into reinforcement learning (RL), converts it into *hierarchical* reinforcement learning (HRL).

Update: Jurgen Schmidhuber kindly suggested some corrections concerning the early work on intrinsic motivation, subgoal discovery and artificial curiosity since 1990, which I have incorporated and expanded. Suppose your friend just baked and shared an excellent cake with you, and you would like to know its recipe. It might seem that it should be very easy for your friend to just tell you how to cook the cake — that it should be easy for him to get across the recipe. But this is a subtler task than you might think; how detailed should the instructions be? Does the friend have to explain in deta

saved by

• Daniel Bashir
• Fernando Silva
• Nhật Hoàng Nguyễn Phan

related reading

• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Hierarchical Reinforcement Learning | Towards Data Sciencetowardsdatascience.com
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective - PMCncbi.nlm.nih.gov
• Deep Reinforcement Learning from Human Preferencesproceedings.neurips.cc
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• Successor Features for Transfer in Reinforcement Learningproceedings.neurips.cc
• [1606.05312] Successor Features for Transfer in Reinforcement Learningarxiv.org
• [2010.15638] Abstract Value Iteration for Hierarchical Reinforcement Learningarxiv.org
• Learning Beyond Gradientstrinkle23897.github.io
• Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionsarxiv.org
• Contrastive Learning as Goal-Conditioned Reinforcement Learningarxiv.org