DSLT 0. Distilling Singular Learning Theory — LessWrong
TLDR; In this sequence I distill Sumio Watanabe's Singular Learning Theory (SLT) by explaining the essence of its main theorem - Watanabe's Free Ener…
x DSLT 0. Distilling Singular Learning Theory — LessWrong Distilling Singular Learning Theory Singular Learning Theory Interpretability (ML & AI) Logic & Mathematics Probability & Statistics AI Frontpage 96 DSLT 0. Distilling Singular Learning Theory by Liam Carroll 16th Jun 2023 AI Alignment Forum 7 min read 8 96 Ω 29 TLDR; In this sequence I distill Sumio Watanabe's Singular Learning Theory (SLT) by explaining the essence of its main theorem - Watanabe's Free Energy Formula for Singular Models - and illustrating its implications with intuition-building examples. I then show why neural networ
Explore this link on the map →saved by
related reading
- Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
- Distilling Singular Learning Theory — LessWronglesswrong.com
- On neural scaling and the quanta hypothesisericjmichaud.com
- AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
- Neural networks generalize because of this one weird trick — LessWronglesswrong.com
- Neural networks generalize because of this one weird trick — AI Alignment Forumalignmentforum.org
- DSLT 2. Why Neural Networks obey Occam's Razor — LessWronglesswrong.com
- Timaeus | Learn about SLTtimaeus.co
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- DSLT 1. The RLCT Measures the Effective Dimension of Neural Networks — LessWronglesswrong.com
- Growth and Form in a Toy Model of Superposition — LessWronglesswrong.com
- DSLT 3. Neural Networks are Singular — LessWronglesswrong.com