Distributed Representations: Composition & Superposition
An informal note on the relationship between superposition and distributed representations by Chris Olah. Published May 4th, 2023. Not published yet. No DOI yet. Distributed representations are a classic idea in both neuroscience and connectionist approaches to AI. We're often asked how our work on superposition relates to it. Since publishing our original paper on superposition, we've had more time to reflect on the relationship between the topics and discuss it with people, and wanted to expand on our earlier discussion in the related work section and share a few thoughts. (We care a lot about superposition and the structure of distributed representations because decomposing representations into independent components is necessary to escape the curse of dimensionality and understand neural networks.) It seems to us that "distributed representations" might be understood as containing two different ideas, which we'll call "composition" and "superposition".These two different notions of
Distributed Representations: Composition & Superposition Transformer Circuits Thread Distributed Representations: Composition & Superposition An informal note on the relationship between superposition and distributed representations by Chris Olah. Published May 4th, 2023. Distributed representations are a classic idea in both neuroscience and connectionist approaches to AI. We're often asked how our work on superposition relates to it. Since publishing our original paper on superposition, we've had more time to reflect on the relationship between the topics and discuss it with people, and want
Explore this link on the map →related reading
- Toy Models of Superpositiontransformer-circuits.pub
- Interpretability Dreamstransformer-circuits.pub
- What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
- Zoom In: An Introduction to Circuitsdistill.pub
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
- SAE feature geometry is outside the superposition hypothesis — LessWronglesswrong.com
- Superposition, Memorization, and Double Descenttransformer-circuits.pub
- Circuits in Superposition 2: Now with Less Wrong Math — LessWronglesswrong.com
- how neural networks think at scalemarmik.xyz
- Circuits Updates — May 2023transformer-circuits.pub
- SAE feature geometry is outside the superposition hypothesis — AI Alignment Forumalignmentforum.org
- Theoretical Motivations for Deep Learning | Rinu Boneyrinuboney.github.io