Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning | HTML5

Existing offline hierarchical reinforcement learning methods rely on high-level policy learning to generate subgoal sequences. However, their efficiency degrades as task horizons increase, and they lack effective strategies for stitching useful state transitions across different trajectories. We propose Graph-Assisted Stitching (GAS), a novel framework that formulates subgoal selection as a graph search problem rather than learning an explicit high-level policy. By embedding states into a Temporal Distance Representation (TDR) space, GAS clusters semantically similar states from different trajectories into unified graph nodes, enabling efficient transition stitching. A shortest-path algorithm is then applied to select subgoal sequences within the graph, while a low-level policy learns to reach the subgoals. To improve graph quality, we introduce the Temporal Efficiency (TE) metric, which filters out noisy or inefficient transition states, significantly enhancing task performance. GAS o

Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning Seungho Baek Taegeon Park Jongchan Park Seungjun Oh Yusung Kim Abstract Existing offline hierarchical reinforcement learning methods rely on high-level policy learning to generate subgoal sequences. However, their efficiency degrades as task horizons increase, and they lack effective strategies for stitching useful state transitions across different trajectories. We propose Graph-Assisted Stitching (GAS), a novel framework that formulates subgoal selection as a graph search problem rather than learning an explicit high-le

related reading

• The Promise of Hierarchical Reinforcement Learningthegradient.pub
• Should I Use Offline RL or Imitation Learning? – The Berkeley Artificial Intelligence Research Blogbair.berkeley.edu
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Q-learning is not yet scalableseohong.me
• [2010.15638] Abstract Value Iteration for Hierarchical Reinforcement Learningarxiv.org
• Contrastive Learning as Goal-Conditioned Reinforcement Learningarxiv.org
• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io
• [2103.06326] S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningarxiv.org
• A General Goal-Conditioned Minecraft Model - Pantographpantograph.com
• Progressive Point Matchingprestonfu.com
• Enhancing Offline Reinforcement Learning with Curriculum Learning-Based Trajectory Valuationarxiv.org
• 2106.01345arxiv.org