Why Local Minima Are Rare in High-Dimensional Random Landscapes

N/A

Why Local Minima Are Rare in High-Dimensional Landscapes A note on Hessians, saddles, and Morse critical points Grant Stenger May 17, 2025 Abstract A common slogan in high-dimensional optimization is that local minima are rare while saddle points are abundant. This slogan is not literally true for every smooth function: convex functions, for example, may have a unique global minimum and no saddles at all. A…

saved by

• Carmen Gutierrez

related reading

• Hessian matrix - Wikipediaen.wikipedia.org
• Pen and Paper Exercises in Machine Learningarxiv.org
• Gregory Gundersengregorygundersen.com
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
• 1412.0233arxiv.org
• Musings on typicality – Sander Dielemansander.ai
• Why Deep Learning Works – Key Insights and Saddle Points - KDnuggetskdnuggets.com
• Thoughts on loss landscapes and why deep learning worksberen.io
• Thoughts on Loss Landscapes and why Deep Learning works — LessWronglesswrong.com
• [1605.08101] Global rates of convergence for nonconvex optimization on manifoldsarxiv.org
• An optimization perspective on log-concave sampling and beyond | Sinho Chewichewisinho.github.io
• Nicholas Cook - Duke Mathsites.math.duke.edu