Growth and Form in a Toy Model of Superposition — LessWrong

> TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developme…

x Growth and Form in a Toy Model of Superposition — LessWrong Singular Learning Theory Interpretability (ML & AI) Superposition AI World Modeling Frontpage 92 Growth and Form in a Toy Model of Superposition by Liam Carroll , Edmund Lau 8th Nov 2023 AI Alignment Forum 17 min read 7 92 Ω 33 TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developmental stages of the Toy Model of Superposition, understanding growth and form from the perspective of Singular Learning Theory (SLT). Ernst Haeckel's K u nstformen

saved by

• Sam B
• Arya Pasumarthi

related reading

• Timaeus | Learn about SLTtimaeus.co
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• DSLT 0. Distilling Singular Learning Theory — LessWronglesswrong.com
• Toy Models of Superpositiontransformer-circuits.pub
• On neural scaling and the quanta hypothesisericjmichaud.com
• Neural networks generalize because of this one weird trick — LessWronglesswrong.com
• Neural networks generalize because of this one weird trick — AI Alignment Forumalignmentforum.org
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
• The generalization phase diagram — LessWronglesswrong.com
• Crystals in NNs: Technical Companion Piece — LessWronglesswrong.com
• [2603.01192] A Basin-Selection Perspective on Grokking via Singular Learning Theoryarxiv.org