The Promise of Hierarchical Reinforcement Learning

This idea of temporal abstraction, once incorporated into reinforcement learning (RL), converts it into *hierarchical* reinforcement learning (HRL).

Update: Jurgen Schmidhuber kindly suggested some corrections concerning the early work on intrinsic motivation, subgoal discovery and artificial curiosity since 1990, which I have incorporated and expanded. Suppose your friend just baked and shared an excellent cake with you, and you would like to know its recipe. It might seem that it should be very easy for your friend to just tell you how to cook the cake — that it should be easy for him to get across the recipe. But this is a subtler task than you might think; how detailed should the instructions be? Does the friend have to explain in deta

saved by

• Daniel Bashir
• Fernando Silva

related reading

• Hierarchical Reinforcement Learning | Towards Data Sciencetowardsdatascience.com
• Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective - PMCncbi.nlm.nih.gov
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• [2010.15638] Abstract Value Iteration for Hierarchical Reinforcement Learningarxiv.org
• Q-learning is not yet scalableseohong.me
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Key Papers in Deep RL - Spinning Up documentationspinningup.openai.com
• pdfopenreview.net
• [2506.07744] Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learningar5iv.labs.arxiv.org
• Evolution as Backstop for Reinforcement Learning · Gwern.netgwern.net
• Reinforcement learning - Wikipediaen.wikipedia.org
• Seohong Park on X: "We scaled up an "alternative" paradigm in RL: *divide and conquer*. Compared to Q-learning (TD learning), divide and conquer can naturally scale to much longer horizons. Blog post: https://t.co/xtXBzya0bI Paper: https://t.co/nqYkLucsWu ↓ https://t.co/XCdgUzaLxF" / Xx.com