Reinforcement Learning with Action Chunking

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on. Authors: achieve the best HTML results from your LaTeX submissions by following these best practices. We present Q-chunking, a simple yet effective recipe for improving reinforcement learning (RL) algorithms for long-horizon, sparse-reward tasks. Our recipe is designed for the offline-to-online RL setting, where the goal is to leverage an offline prior dataset to maximize the sample-efficiency of online learning. Effective exploration and sample-efficient learning remain cent

Reinforcement Learning with Action Chunking Qiyang Li, Zhiyuan Zhou, Sergey Levine UC Berkeley {qcli,zhiyuan_zhou,svlevine}@eecs.berkeley.edu Abstract We present Q-chunking , a simple yet effective recipe for improving reinforcement learning (RL) algorithms for long-horizon, sparse-reward tasks. Our recipe is designed for the offline-to-online RL setting, where the goal is to leverage an offline prior dataset to maximize the sample-efficiency of online learning. Effective exploration and sample-efficient learning remain central challenges in this setting, as it is not obvious how the offline d

related reading

• [2507.07969] Reinforcement Learning with Action Chunkingar5iv.labs.arxiv.org
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Q-learning is not yet scalableseohong.me
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• The 37 Implementation Details of Proximal Policy Optimization · The ICLR Blog Trackiclr-blog-track.github.io
• The Promise of Hierarchical Reinforcement Learningthegradient.pub
• [2005.01643] Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemsar5iv.labs.arxiv.org
• Seohong Park on X: "We scaled up an "alternative" paradigm in RL: *divide and conquer*. Compared to Q-learning (TD learning), divide and conquer can naturally scale to much longer horizons. Blog post: https://t.co/xtXBzya0bI Paper: https://t.co/nqYkLucsWu ↓ https://t.co/XCdgUzaLxF" / Xx.com
• Deep Q-Networks Explained — LessWronglesswrong.com
• An Updated Introduction to Reinforcement Learning | Sri's Blogsrianumakonda.com
• Exploration for the Efficient Deployment of Reinforcement Learning Agentsopenreview.net
• State of RL for reasoning LLMs | A. Weersaweers.de