RL without TD learning – The Berkeley Artificial Intelligence Research Blog

The BAIR Blog

In this post, I’ll introduce a reinforcement learning (RL) algorithm based on an “alternative” paradigm: divide and conquer . Unlike traditional methods, this algorithm is not based on temporal difference (TD) learning (which has scalability challenges ), and scales well to long-horizon tasks. We can do Reinforcement Learning (RL) based on divide and conquer, instead of temporal difference (TD) learning. Problem setting: off-policy RL Our problem setting is off-policy RL . Let’s briefly review what this means. There are two classes of algorithms in RL: on-policy RL and off-policy RL. On-policy

related reading

• RL without TD learningseohong.me
• Seohong Park on X: "We scaled up an "alternative" paradigm in RL: *divide and conquer*. Compared to Q-learning (TD learning), divide and conquer can naturally scale to much longer horizons. Blog post: https://t.co/xtXBzya0bI Paper: https://t.co/nqYkLucsWu ↓ https://t.co/XCdgUzaLxF" / Xx.com
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Q-learning is not yet scalableseohong.me
• Scaling Laws for Value-Based RLvalue-scaling.github.io
• The Promise of Hierarchical Reinforcement Learningthegradient.pub
• RLAlgsInMDPs.pdfsites.ualberta.ca
• RUDDER - Reinforcement Learning with Delayed Rewards | rudderml-jku.github.io
• Value-Based Deep RL Scales Predictablyarxiv.org
• An Updated Introduction to Reinforcement Learning | Sri's Blogsrianumakonda.com
• Reinforcement Learning in Newcomblike Problemsproceedings.neurips.cc
• Key Papers in Deep RL - Spinning Up documentationspinningup.openai.com