Should I Use Offline RL or Imitation Learning? – The Berkeley Artificial Intelligence Research Blog

The BAIR Blog

Figure 1: Summary of our recommendations for when a practitioner should BC and various imitation learning style methods, and when they should use offline RL approaches. Offline reinforcement learning allows learning policies from previously collected data, which has profound implications for applying RL in domains where running trial-and-error learning is impractical or dangerous, such as safety-critical settings like autonomous driving or medical treatment planning. In such scenarios, online exploration is simply too risky, but offline RL methods can learn effective policies from logged data

saved by

• Fernando Silva
• Karan Dalal

related reading

• Learning to Imitate | SAIL Blogai.stanford.edu
• [2204.05618] When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?ar5iv.labs.arxiv.org
• [2406.09329] Is Value Learning Really the Main Bottleneck in Offline RL?ar5iv.labs.arxiv.org
• 2109.10813.pdfarxiv.org
• Reinforcement Learning via Implicit Imitation Guidancearxiv.org
• [2103.06326] S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningarxiv.org
• State of Robot Learning, December 2025vedder.io
• Life lessons from reinforcement learning - Jason Weijasonwei.net
• Ch. 21 - Imitation Learningunderactuated.mit.edu
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Imitation Bootstrapped Reinforcement Learningarxiv.org
• [2005.01643] Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemsar5iv.labs.arxiv.org