Thoughts on Loss Landscapes and why Deep Learning works — LessWrong

There are essentially two fundamental questions in the science of deep learning: 1.) Why are models trainable? And 2.) Why do models generalize? The…

x Thoughts on Loss Landscapes and why Deep Learning works — LessWrong General intelligence Machine Learning (ML) AI Frontpage 54 Thoughts on Loss Landscapes and why Deep Learning works by beren 25th Jul 2023 21 min read 4 54 Crossposted from my personal blog . Epistemic status : Pretty uncertain. I don’t have an expert level understanding of current views in the science of deep learning about why optimization works but just read papers as an amateur. Some of the arguments I present here might be already either known or disproven. If so please let me know! There are essentially two fundamental

related reading

• Thoughts on loss landscapes and why deep learning worksberen.io
• The Little Book of Deep Learningfleuret.org
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
• The Decade of Deep Learning | Leo Gaobmk.sh
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
• Why Deep Learning Works – Key Insights and Saddle Points - KDnuggetskdnuggets.com
• Elon Litman | Elements of a Vector Spaceelonlit.com
• Theoretical Motivations for Deep Learning | Rinu Boneyrinuboney.github.io
• The Generalization Mystery: Sharp vs Flat Minimainference.vc