A (dis)analogy for (mis)understanding the lottery ticket hypothesis - Vaishnavh Nagarajan
vaishnavh.github.io · 1,456 words · saved by 1 readers
“For any given task, a sufficiently large neural network will contain within it a good subnetwork that can be isolate...
A (dis)analogy for (mis)understanding the lottery ticket hypothesis Published on June 23, 2026 • 8 MIN READ “For any given task, a sufficiently large neural network will contain within it a good subnetwork that can be isolated and finetuned to solve the given task”, is how we would recall the Lottery Ticket Hypothesis (LTH) (Frankle and Carbin, 2018)1, but this statement by itself does not specify how large the parent network must be. So, it may be tempting to conclude that the parent network must be really, really large—even exponentially large—for at least a slim chance that some…
saved by
related reading
- Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu
- Toy Models of Superpositiontransformer-circuits.pub
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
- Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
- Neural networks are not already cleanlesswrong.com
- Human-like Neural Nets by Catapulting · Gwern.netgwern.net
- Understanding the Neural Tangent Kernel – EigenTaleseigentales.com
- What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
- [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
- The Hardware Lotteryhardwarelottery.github.io
- nn-notes.pdfboris-hanin.github.io