Google AI Blog: A New Lens on Understanding Generalization in Deep Learning

Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University Understanding generalization is one of the fundamental unsolved pr...

A New Lens on Understanding Generalization in Deep Learning Skip to main content A New Lens on Understanding Generalization in Deep Learning March 10, 2021 Hanie Sedghi, Google Research and Preetum Nakkiran, Harvard University Quick links Share Copy link × Understanding generalization is one of the fundamental unsolved problems in deep learning. Why does optimizing a model on a finite set of training data lead to good performance on a held-out test set ? This problem has been studied extensively in machine learning, with a rich history going back more than 50 years. There are now many mathemat

related reading

• Just Ask for Generalization | Eric Jangevjang.com
• The Little Book of Deep Learningfleuret.org
• The Scaling Hypothesis · Gwern.netgwern.net
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• [2503.02113] Deep Learning is Not So Mysterious or Differentarxiv.org
• The Decade of Deep Learning | Leo Gaobmk.sh
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• Clare Lyle | What's grokking good for?clarelyle.com
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• 3.6. Generalization — Dive into Deep Learning 1.0.3 documentationd2l.ai
• arxiv.org/pdf/1805.08522arxiv.org