Growth and Form in a Toy Model of Superposition — LessWrong
lesswrong.com · 5,972 words · saved by 2 readers
> TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developme…
x Growth and Form in a Toy Model of Superposition — LessWrong Singular Learning Theory Interpretability (ML & AI) Superposition AI World Modeling Frontpage 92 Growth and Form in a Toy Model of Superposition by Liam Carroll , Edmund Lau 8th Nov 2023 AI Alignment Forum 17 min read 7 92 Ω 33 TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developmental stages of the Toy Model of Superposition, understanding growth and form from the perspective of Singular Learning Theory (SLT). Ernst Haeckel's K u nstformen
saved by
related reading
- Timaeus | Learn about SLTtimaeus.co
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
- DSLT 0. Distilling Singular Learning Theory — LessWronglesswrong.com
- Toy Models of Superpositiontransformer-circuits.pub
- Neural networks generalize because of this one weird trick — LessWronglesswrong.com
- On neural scaling and the quanta hypothesisericjmichaud.com
- Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
- Neural networks generalize because of this one weird trick — AI Alignment Forumalignmentforum.org
- [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
- [2608.13335] Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Lawsarxiv.org
- 2309.07311.pdfarxiv.org