Stochastic Spectral Descent for Restricted Boltzmann Machines
Restricted Boltzmann Machines (RBMs) are widely used as building blocks for deep learning models. Learning typically proceeds by using stochastic gradient descent, and the gradients are estimated w...
Stochastic Spectral Descent for Restricted Boltzmann Machines David Carlson, Volkan Cevher, Lawrence Carin Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics , PMLR 38:111-119, 2015. Abstract Restricted Boltzmann Machines (RBMs) are widely used as building blocks for deep learning models. Learning typically proceeds by using stochastic gradient descent, and the gradients are estimated with sampling methods. However, the gradient estimation is a computational bottleneck, so better use of the gradients will speed up the descent algorithm. To this end
Explore this link on the map →related reading
- Stochastic gradient descent - Wikipediaen.m.wikipedia.org
- The Little Book of Deep Learningfleuret.org
- Why Momentum Really Worksdistill.pub
- A Visual Explanation of Gradient Descent Methods (Momentum, AdaGrad, RMSProp, Adam) | Towards Data Sciencetowardsdatascience.com
- [1512.04202] Preconditioned Stochastic Gradient Descentarxiv.org
- [2101.12176] On the Origin of Implicit Regularization in Stochastic Gradient Descentarxiv.org
- Competing with sampling — Alignment Research Centeralignment.org
- [2503.22478] Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descentarxiv.org
- Notes on the Origin of Implicit Regularization in SGDinference.vc
- ARC progress update: Competing with sampling — LessWronglesswrong.com
- Deep Energy Estimator.pdfarxiv.org
- Does Muon improve regulatory DNA learning? Part 1. — Origin Bioorigin.bio