The generalization phase diagram — LessWrong
Introduction This is part I of 3.5 planned posts on the “tempered posterior” (one of which will be “SLT in a nutshell”). It is a kind of moral sequel…
x The generalization phase diagram — LessWrong Interpretability (ML & AI) Singular Learning Theory AI Frontpage 28 The generalization phase diagram by Dmitry Vaintrob 26th Jan 2025 20 min read 2 28 Introduction This is part I of 3.5 planned posts on the “tempered posterior” (one of which will be “SLT in a nutshell”). It is a kind of moral sequel to “ Dmitry’s Koan ” and “ Logits, log-odds, and loss for parallel circuits ”, and is related to the post on grammars . It can be read independently of any of these. In my view, one of the most valuable contributions of singular learning theory (SLT) s
saved by
related reading
- The nature of LLM algorithmic progress (v2) — LessWronglesswrong.com
- arxiv.org/pdf/1805.08522arxiv.org
- Simulators — LessWronglesswrong.com
- The Local Learning Coefficient: A Singularity-Aware Complexity Measurearxiv.org
- Just Ask for Generalization | Eric Jangevjang.com
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- Deep learning as program synthesis — LessWronglesswrong.com
- interpreting GPT: the logit lens — LessWronglesswrong.com
- Growth and Form in a Toy Model of Superposition — LessWronglesswrong.com
- [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
- Neural networks generalize because of this one weird trick — LessWronglesswrong.com
- DSLT 0. Distilling Singular Learning Theory — LessWronglesswrong.com