Double descent in human learning · Chris Said

The uncanny resemblance between double descent and a 50 year old theory from psychology

Double descent in human learning · Chris Said --> Chris Said I am a data scientist at Propel. This blog is mostly about statistics, technology, and science. Double descent in human learning 21 Apr 2023 In machine learning, double descent is a surprising phenomenon where increasing the number of model parameters causes test performance to get better, then worse, and then better again. It refutes the classical overfitting finding that if you have too many parameters in your model, your test error will always keep getting worse with more parameters. For a surprisingly wide range of models

related reading

• [2303.14151] Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzlearxiv.org
• Understanding “Deep Double Descent” — LessWronglesswrong.com
• Deep Double Descent (cross-posted on OpenAI blog) – Windows On Theorywindowsontheory.org
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• Lil'Loglilianweng.github.io
• The Scaling Hypothesis · Gwern.netgwern.net
• Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blogsohl-dickstein.github.io
• Just Ask for Generalization | Eric Jangevjang.com
• Human-like Neural Nets by Catapulting · Gwern.netgwern.net
• 4 – The Overfitting Iceberg – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
• You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
• Deep Learning is Not So Mysterious or Differentarxiv.org