On neural scaling and the quanta hypothesis
Several years ago, humanity began an experiment. This experiment is now on track to become one of the most expensive ever attempted. We don't yet know what the results of this experiment will be. Certain results could be transformative. The experiment could not only alter the dynamics of wealth and power in human life, but change our basic status as a species: it could put us in contact with alien minds more capable than our own for the first time. Alternatively, some still predict that the experiment will be largely a waste of time and energy. Despite its importance, we don't have a mature theory backing this experiment. And the number of people working on such a theory in public is relatively small.
On neural scaling and the quanta hypothesis --> On neural scaling and the quanta hypothesis 2026-01-12 Testing testing testing. --> Several years ago, humanity began an experiment. This experiment is now on track to become one of the most expensive ever attempted. We don't yet know what the results of this experiment will be. Certain results could be transformative. The experiment could not only alter the dynamics of wealth and power in human life, but change our basic status as a species: it could put us in contact with alien minds more capable than our own for the first time. Alternatively,
saved by
- Tasha Pais
- Hangyul Lyna Kim
- Asher P
- Sudarsh K
- Paul Scotti
- Dhruv Sheth
- Vincent Huang
- Akira Yoshiyama
- Davis Brown
- Sheikh Abdur Raheem Ali
related reading
- A short note on interpretability and mindsericjmichaud.com
- Scaling Laws, Carefully | Lil'Loglilianweng.github.io
- The Scaling Hypothesis · Gwern.netgwern.net
- [2303.13506] The Quantization Model of Neural Scalingarxiv.org
- Scaling is subtler than it seemsberen.io
- [2608.13335] Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Lawsarxiv.org
- Will scaling work?dwarkeshpatel.com
- The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
- [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
- Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu
- Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
- Quantization from the ground upngrok.com