Hierarchical Reinforcement Learning | by Ankita Sinha | Towards Data Science

Hierarchical Reinforcement Learning decomposes long horizon decision making process into simpler sub-tasks. This idea is very similar to breaking down large number of lines of code to smaller functions each performing a very specific task. Let’s look at an example, Suppose the agent has to clear or set a dining table. This includes the task of reaching and grasping dishes. These are high level tasks. On a lower level, it requires the task of controlling and moving the limbs and then the fingers to reach out and grasp objects and subsequently put them in the proper place. Hierarchical Reinforcement Learning is designed with the same logic. There are multiple levels of policies with each policy handling a lower level task like moving the fingers and the higher level policies handling tasks like grasping the objects. HRL gives us multiple benefits during training and exploration: A few common architectures for HRL are- Lets look at how to build your own Option-Critic framework in a simple

Hierarchical Reinforcement Learning | Towards Data Science Skip to content Machine Learning Hierarchical Reinforcement Learning With Options-Critic framework using tabular Q-Learning Ankita Sinha Feb 11, 2022 4 min read Share unsplash.com Hierarchical Reinforcement Learning decomposes long horizon decision making process into simpler sub-tasks. This idea is very similar to breaking down large number of lines of code to smaller functions each performing a very specific task. Let’s look at an example, Suppose the agent has to clear or set a dining table. This includes the task of reaching

related reading

• The Promise of Hierarchical Reinforcement Learningthegradient.pub
• Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective - PMCncbi.nlm.nih.gov
• [2010.15638] Abstract Value Iteration for Hierarchical Reinforcement Learningarxiv.org
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Q-learning is not yet scalableseohong.me
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• pdfopenreview.net
• Learning Beyond Gradientstrinkle23897.github.io
• Seohong Park on X: "We scaled up an "alternative" paradigm in RL: *divide and conquer*. Compared to Q-learning (TD learning), divide and conquer can naturally scale to much longer horizons. Blog post: https://t.co/xtXBzya0bI Paper: https://t.co/nqYkLucsWu ↓ https://t.co/XCdgUzaLxF" / Xx.com
• Key Papers in Deep RL - Spinning Up documentationspinningup.openai.com
• [2506.07744] Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learningar5iv.labs.arxiv.org