Should I Use Offline RL or Imitation Learning? – The Berkeley Artificial Intelligence Research Blog

The BAIR Blog

Figure 1: Summary of our recommendations for when a practitioner should BC and various imitation learning style methods, and when they should use offline RL approaches. Offline reinforcement learning allows learning policies from previously collected data, which has profound implications for applying RL in domains where running trial-and-error learning is impractical or dangerous, such as safety-critical settings like autonomous driving or medical treatment planning. In such scenarios, online exploration is simply too risky, but offline RL methods can learn effective policies from logged data

saved by

• Fernando Silva

related reading

• [2204.05618] When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?ar5iv.labs.arxiv.org
• The Promise of Hierarchical Reinforcement Learningthegradient.pub
• [2406.09329] Is Value Learning Really the Main Bottleneck in Offline RL?ar5iv.labs.arxiv.org
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Reinforcement Learning via Implicit Imitation Guidancearxiv.org
• State of Robot Learning, December 2025vedder.io
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Imitation Bootstrapped Reinforcement Learningarxiv.org
• [2005.01643] Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemsar5iv.labs.arxiv.org
• Learning to Imitate | SAIL Blogai.stanford.edu
• Ch. 21 - Imitation Learningunderactuated.mit.edu
• Exploration for the Efficient Deployment of Reinforcement Learning Agentsopenreview.net