A (dis)analogy for (mis)understanding the lottery ticket hypothesis - Vaishnavh Nagarajan
vaishnavh.github.io · 1,456 words · saved by 1 readers
“For any given task, a sufficiently large neural network will contain within it a good subnetwork that can be isolate...
A (dis)analogy for (mis)understanding the lottery ticket hypothesis Published on June 23, 2026 • 8 MIN READ “For any given task, a sufficiently large neural network will contain within it a good subnetwork that can be isolated and finetuned to solve the given task”, is how we would recall the Lottery Ticket Hypothesis (LTH) (Frankle and Carbin, 2018)1, but this statement by itself does not specify how large the parent network must be. So, it may be tempting to conclude that the parent network must be really, really large—even exponentially large—for at least a slim chance that some…
saved by
related reading
- On neural scaling and the quanta hypothesisericjmichaud.com
- Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu
- Zoom In: An Introduction to Circuitsdistill.pub
- Toy Models of Superpositiontransformer-circuits.pub
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- nn-notes.pdfboris-hanin.github.io
- The Scaling Hypothesis · Gwern.netgwern.net
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- Fact Finding: Attempting to Reverse-Engineer Factual Recall on the Neuron Level (Post 1) — AI Alignment Forumalignmentforum.org
- Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
- [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org