Understanding “Deep Double Descent” — LessWrong

If you're not familiar with the double descent phenomenon, I think you should be. I consider double descent to be one of the most interesting and sur…

x Understanding “Deep Double Descent” — LessWrong Best of LessWrong 2019 Double Descent Lottery Ticket Hypothesis Machine Learning (ML) AI Frontpage 157 Understanding “Deep Double Descent” by evhub 6th Dec 2019 AI Alignment Forum 6 min read 51 157 Ω 54 If you're not familiar with the double descent phenomenon, I think you should be. I consider double descent to be one of the most interesting and surprising recent results in analyzing and understanding modern machine learning. Today, Preetum et al. released a new paper, “ Deep Double Descent ,” which I think is a big further advancement in our

related reading

• Deep Double Descent (cross-posted on OpenAI blog) – Windows On Theorywindowsontheory.org
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• Double descent in human learning · Chris Saidchris-said.io
• The Little Book of Deep Learningfleuret.org
• [2503.02113] Deep Learning is Not So Mysterious or Differentarxiv.org
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• The Scaling Hypothesis · Gwern.netgwern.net
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• 4 – The Overfitting Iceberg – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
• Clare Lyle | What's grokking good for?clarelyle.com
• Superposition, Memorization, and Double Descenttransformer-circuits.pub
• The Decade of Deep Learning | Leo Gaobmk.sh