When is Realizability Sufficient for Off-Policy Reinforcement Learning? | HTML5

Understanding when reinforcement learning algorithms can make successful off-policy predictions—and when the may fail to do so–remains an open problem. Typically, model-free algorithms for reinforcement learning are an…

[2211.05311] When is Realizability Sufficient for Off-Policy Reinforcement Learning? When is Realizability Sufficient for Off-Policy Reinforcement Learning? Andrea Zanette Abstract Understanding when reinforcement learning algorithms can make successful off-policy predictions—and when the may fail to do so–remains an open problem. Typically, model-free algorithms for reinforcement learning are analyzed under a condition called Bellman completeness when they operate off-policy with function approximation, unless additional conditions are met. However, Bellman completeness is a requirement that

related reading

• Reinforcement Learning in Newcomblike Problemsproceedings.neurips.cc
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• [2602.11399] Can We Really Learn One Representation to Optimize All Rewards?arxiv.org
• [2307.04354] Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Dataar5iv.labs.arxiv.org
• 2203.00543arxiv.org
• 2208.05129arxiv.org
• c21f4ce780c5c9d774f79841b81fdc6d-Paper.pdfproceedings.neurips.cc
• [2506.22401] Exploration from a Primal-Dual Lens: Value-Incentivized Actor-Critic Methods for Sample-Efficient Online RLarxiv.org
• [2007.08202] Provably Good Batch Reinforcement Learning Without Great Explorationar5iv.labs.arxiv.org
• [2507.13181] Spectral Bellman Method: Unifying Representation and Exploration in RLarxiv.org
• [2306.01157] Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confoundingarxiv.org
• [1905.13341] On Value Functions and the Agent-Environment Boundaryar5iv.labs.arxiv.org