Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations

N/A

Journal of Machine Learning Research 20 (2019) 1-47 Submitted 9/17; Revised 2/19; Published 3/19 Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations Qianxiao Li liqix@ihpc.a-star.edu.sg Institute of High Performance Computing Agency for Science, Technology and Research 1 Fusionopolis Way, Connexis North, Singapore 138632 Cheng Tai chengtai@pku.edu.cn Beijing Institute of Big Data…

saved by

• Roland Riachi

related reading

• Scalable Gradients for Stochastic Differential Equationsarxiv.org
• A Primer on Stochastic Partial Differential Equationsmath.utah.edu
• Stochastic Heat Equations with General Multiplicative Gaussian Noises: Hölder Continuity and Intermittencyarxiv.org
• [1511.06251] Stochastic modified equations and adaptive stochastic gradient algorithmsarxiv.org
• Stochastic gradient descent - Wikipediaen.m.wikipedia.org
• Stochastic Partial Differential Equations: Classical and Newmi.fu-berlin.de
• Why Momentum Really Worksdistill.pub
• [2101.12176] On the Origin of Implicit Regularization in Stochastic Gradient Descentarxiv.org
• Stochastic Gradient Descent: An intuitive proofmedium.com
• Keep the gradient flowingfa.bianp.net
• Policy Gradient Algorithms | Lil'Loglilianweng.github.io
• [1512.04202] Preconditioned Stochastic Gradient Descentarxiv.org