Learning Mechanics

The mathematical science of neural network training

Apr 23, 2026 Deep linear networks are a surprisingly useful toy model of weight-space dynamics Deep linear networks are simple enough to study analytically but rich enough to exhibit key phenomena of neural network training. Apr 23, 2026 On neural scaling and the quanta hypothesis Eric J. Michaud What is the origin of neural scaling laws? What do they tell us about the structure of data? What are the limits of interpretability? Apr 23, 2026 Perspectives on the science of deep learning Three essays on building theory that matters. Coming soon... A visual guide to progressive…

saved by

• Alima Deen

related reading

• A scientific theory of deep learning is emerging, slowly but surely.learningmechanics.pub
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• On neural scaling and the quanta hypothesisericjmichaud.com
• nn-notes.pdfboris-hanin.github.io
• Zoom In: An Introduction to Circuitsdistill.pub
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• [2608.13335] Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Lawsarxiv.org
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• The Scaling Hypothesis · Gwern.netgwern.net
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• Homemrkdh16.github.io