On neural scaling and the quanta hypothesis

Several years ago, humanity began an experiment. This experiment is now on track to become one of the most expensive ever attempted. We don't yet know what the results of this experiment will be. Certain results could be transformative. The experiment could not only alter the dynamics of wealth and power in human life, but change our basic status as a species: it could put us in contact with alien minds more capable than our own for the first time. Alternatively, some still predict that the experiment will be largely a waste of time and energy. Despite its importance, we don't have a mature theory backing this experiment. And the number of people working on such a theory in public is relatively small.

On neural scaling and the quanta hypothesis --> On neural scaling and the quanta hypothesis 2026-01-12 Testing testing testing. --> Several years ago, humanity began an experiment. This experiment is now on track to become one of the most expensive ever attempted. We don't yet know what the results of this experiment will be. Certain results could be transformative. The experiment could not only alter the dynamics of wealth and power in human life, but change our basic status as a species: it could put us in contact with alien minds more capable than our own for the first time. Alternatively,

saved by

• Tasha Pais
• Hangyul Lyna Kim
• Asher P
• Sudarsh K
• Paul Scotti
• Dhruv Sheth
• Vincent Huang
• Akira Yoshiyama
• Davis Brown
• Sheikh Abdur Raheem Ali

related reading

• The Scaling Hypothesis · Gwern.netgwern.net
• [2303.13506] The Quantization Model of Neural Scalingarxiv.org
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• A short note on interpretability and mindsericjmichaud.com
• Language models can explain neurons in language modelsopenaipublic.blob.core.windows.net
• Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu
• Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
• [2602.05970] Inverse Depth Scaling From Most Layers Being Similararxiv.org
• Bits, FLOPS, and Watts: A Systems-Level Perspective of Scaling LLMs — Part 1 | by Asheesh Goja | Mediummedium.com
• The Scaling Hypothesis · Gwern.netgwern.net
• New Scaling Laws for Large Language Models — LessWronglesswrong.com