RL without TD learning – The Berkeley Artificial Intelligence Research Blog

The BAIR Blog

In this post, I’ll introduce a reinforcement learning (RL) algorithm based on an “alternative” paradigm: divide and conquer . Unlike traditional methods, this algorithm is not based on temporal difference (TD) learning (which has scalability challenges ), and scales well to long-horizon tasks. We can do Reinforcement Learning (RL) based on divide and conquer, instead of temporal difference (TD) learning. Problem setting: off-policy RL Our problem setting is off-policy RL . Let’s briefly review what this means. There are two classes of algorithms in RL: on-policy RL and off-policy RL. On-policy

related reading

• RL without TD learningseohong.me
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Q-learning is not yet scalableseohong.me
• Seohong Park on X: "We scaled up an "alternative" paradigm in RL: *divide and conquer*. Compared to Q-learning (TD learning), divide and conquer can naturally scale to much longer horizons. Blog post: https://t.co/xtXBzya0bI Paper: https://t.co/nqYkLucsWu ↓ https://t.co/XCdgUzaLxF" / Xx.com
• Should I Use Offline RL or Imitation Learning? – The Berkeley Artificial Intelligence Research Blogbair.berkeley.edu
• Part 2: Kinds of RL Algorithms - Spinning Up documentationspinningup.openai.com
• Reinforcement Learning in Newcomblike Problemsproceedings.neurips.cc
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• [1802.09081] Temporal Difference Models: Model-Free Deep RL for Model-Based Controlar5iv.labs.arxiv.org
• ML Mentorship: Some Q/A about RLevjang.com
• 2109.10813.pdfarxiv.org
• The Promise of Hierarchical Reinforcement Learningthegradient.pub