4.4 Value Iteration

4.4 Value Iteration

4.4 Value Iteration Next: 4.5 Asynchronous Dynamic Programming Up: 4. Dynamic Programming Previous: 4.3 Policy Iteration Contents 4.4 Value Iteration One drawback to policy iteration is that each of its iterations involves policy evaluation, which may itself be a protracted iterative computation requiring multiple sweeps through the state set. If policy evaluation is done iteratively, then convergence exactly to occurs only in the limit. Must we wait for exact convergence, or can we stop short of that? The example in Figure 4.2 certainly suggests that it may be possible to truncate policy eval

saved by

• Daivik Dhar

related reading

• Variants of Dynamic Programminggoodboychan.github.io
• Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
• Policy Gradient Algorithms | Lil'Loglilianweng.github.io
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• Why Momentum Really Worksdistill.pub
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
• Policy gradient methoden.wikipedia.org
• RLAlgsInMDPs.pdfsites.ualberta.ca
• rltheorybook_ABJKS.pdfrltheorybook.github.io
• Vanilla Policy Gradient - Spinning Up documentationspinningup.openai.com
• Understanding Policy Gradients | John Lambertjohnwlambert.github.io
• Proximal Policy Optimization - Spinning Up documentationspinningup.openai.com