Growth and Form in a Toy Model of Superposition — LessWrong

> TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developme…

x Growth and Form in a Toy Model of Superposition — LessWrong Singular Learning Theory Interpretability (ML & AI) Superposition AI World Modeling Frontpage 92 Growth and Form in a Toy Model of Superposition by Liam Carroll , Edmund Lau 8th Nov 2023 AI Alignment Forum 17 min read 7 92 Ω 33 TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developmental stages of the Toy Model of Superposition, understanding growth and form from the perspective of Singular Learning Theory (SLT). Ernst Haeckel's K u nstformen

saved by

• Sam B
• Arya Pasumarthi

related reading

• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• Timaeus | Learn about SLTtimaeus.co
• Toy Models of Superpositiontransformer-circuits.pub
• Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• The generalization phase diagram — LessWronglesswrong.com
• DSLT 0. Distilling Singular Learning Theory — LessWronglesswrong.com
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• Neural networks generalize because of this one weird trick — LessWronglesswrong.com
• Neural networks generalize because of this one weird trick — AI Alignment Forumalignmentforum.org
• Crystals in NNs: Technical Companion Piece — LessWronglesswrong.com