Google AI Blog: A New Lens on Understanding Generalization in Deep Learning

Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University Understanding generalization is one of the fundamental unsolved pr...

A New Lens on Understanding Generalization in Deep Learning Skip to main content A New Lens on Understanding Generalization in Deep Learning March 10, 2021 Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University Quick links Share Copy link × Understanding generalization is one of the fundamental unsolved problems in deep learning. Why does optimizing a model on a finite set of training data lead to good performance on a held-out test set ? This problem has been studied extensively in machine learning, with a rich history going back more than 50 years. There are now many mathemat

related reading

• Just Ask for Generalization | Eric Jangevjang.com
• Understanding deep learning requires rethinking generalizationarxiv.org
• Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsmathai-iclr.github.io
• The Little Book of Deep Learningfleuret.org
• The Scaling Hypothesis · Gwern.netgwern.net
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• The Decade of Deep Learning | Leo Gaobmk.sh
• [2503.02113] Deep Learning is Not So Mysterious or Differentarxiv.org
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
• Clare Lyle | What's grokking good for?clarelyle.com
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com