3.6. Generalization — Dive into Deep Learning 1.0.0-beta0 documentation
Consider two college students diligently preparing for their final exam. Commonly, this preparation will consist of practicing and testing their abilities by taking exams administered in previous years. Nonetheless, doing well on past exams is no guarantee that they will excel when it matters. For instance, imagine a student, Elephantine Ellie, whose preparation consisted entirely of memorizing the answers to previous years’ exam questions. Even if Ellie were endowed with an elephantine memory, and thus could perfectly recall the answer to any previously seen question, she might nevertheless freeze when faced with a new (previously unseen) question. By comparison, imagine another student, Inductive Irene, with comparably poor memorization skills, but a knack for picking up patterns. Note that if the exam truly consisted of recycled questions from a previous year, Ellie would handily outperform Irene. Even if Irene’s inferred patterns yielded 90% accurate predictions, they could never c
3.6. Generalization — Dive into Deep Learning 1.0.3 documentation 3.6. Generalization ¶ Colab [pytorch] Open the notebook in Colab Colab [mxnet] Open the notebook in Colab Colab [jax] Open the notebook in Colab Colab [tensorflow] Open the notebook in Colab SageMaker Studio Lab Open the notebook in SageMaker Studio Lab Consider two college students diligently preparing for their final exam. Commonly, this preparation will consist of practicing and testing their abilities by taking exams administered in previous years. Nonetheless, doing well on past exams is no guarantee that they will ex
Explore this link on the map →saved by
related reading
- Reshelving generalization - by Ben Recht - arg minargmin.net
- deeplearningbook.org/contents/ml.htmldeeplearningbook.org
- [2503.02113] Deep Learning is Not So Mysterious or Differentarxiv.org
- A Course in Machine Learningciml.info
- Clare Lyle | What's grokking good for?clarelyle.com
- 4 – The Overfitting Iceberg – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
- arxiv.org/pdf/1805.08522arxiv.org
- A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
- Troubles with the Bias-Variance tradeoffejenner.com
- Just Ask for Generalization | Eric Jangevjang.com
- [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
- A Recipe for Training Neural Networkskarpathy.github.io