✳flâneur — a map of the web's best reading

Soft Actor-Critic — Spinning Up documentation

spinningup.openai.com · 2,832 words · saved by 2 readers

Soft Actor Critic (SAC) is an algorithm that optimizes a stochastic policy in an off-policy way, forming a bridge between stochastic policy optimization and DDPG-style approaches. It isn’t a direct successor to TD3 (having been published roughly concurrently), but it incorporates the clipped double-Q trick, and due to the inherent stochasticity of the policy in SAC, it also winds up benefiting from something like target policy smoothing. A central feature of SAC is entropy regularization. The policy is trained to maximize a trade-off between expected return and entropy, a measure of randomness in the policy. This has a close connection to the exploration-exploitation trade-off: increasing entropy results in more exploration, which can accelerate learning later on. It can also prevent the policy from prematurely converging to a bad local optimum. To explain Soft Actor Critic, we first have to introduce the entropy-regularized reinforcement learning setting. In entropy-regularized RL, th

Soft Actor-Critic - Spinning Up documentation --> Docs >> Soft Actor-Critic Edit on GitHub Soft Actor-Critic ¶ Table of Contents Soft Actor-Critic Background Quick Facts Key Equations Entropy-Regularized Reinforcement Learning Soft Actor-Critic Exploration vs. Exploitation Pseudocode Documentation Documentation: PyTorch Version Saved Model Contents: PyTorch Version Documentation: Tensorflow Version Saved Model Contents: Tensorflow Version References Relevant Papers Other Public Implementations Background ¶ (Previously: Background for TD3 ) Soft Actor Critic (SAC) is an algorithm that optimizes

Explore this link on the map →

saved by

related reading