Fit Without Fear: An Over-Fitting Perspective on Modern Deep and Shallow Learning - YouTube

Mikhail Belkin, Ohio State UniversityAbstract:A striking feature of modern supervised machine learning is its pervasive over-parametrization. Deep networks c...

Mikhail Belkin, Ohio State University Abstract: A striking feature of modern supervised machine learning is its pervasive over-parametrization. Deep networks contain millions of parameters, often exceeding the number of data points by orders of magnitude. These networks are trained to nearly interpolate the data by driving the training error to zero. Yet, at odds with most theory, they show excellent test performance. It has become accepted wisdom that these properties are special to deep networks and require non-convex analysis to understand. In this talk I will show that classical…

saved by

• Roi Holtzman

related reading

• Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu
• The necessity of machine learning theory in mitigating AI riskmishabelkin.substack.com
• Bayesian Neural Networkscs.toronto.edu
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• Understanding deep learning requires rethinking generalizationarxiv.org
• Just Ask for Generalization | Eric Jangevjang.com
• The Little Book of Deep Learningfleuret.org
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
• [1706.04599] On Calibration of Modern Neural Networksarxiv.org
• Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
• [2201.02177] Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsarxiv.org
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org