Circuits in Superposition 2: Now with Less Wrong Math — LessWrong

This post is a continuation and clarification of Circuits in Superposition: Compressing many small neural networks into one. That post presented a sketch of a general mathematical framework for compressing different circuits into a network in superposition. On closer inspection, some of it turned out to be wrong, though. The error propagation calculations for networks with multiple layers were incorrect. With the framework used in that post, the errors blow up too much over multiple layers. This post presents a slightly changed construction that fixes those problems, and improves on the original in some other ways as well.[1] By computation in superposition we mean that a network represents features in superposition and performs more computations with them than it has neurons, across multiple layers. Having better models of this is important for understanding how and even if networks use superposition, which in turn is important for mech-interp in general. Performing computation in sup

x Circuits in Superposition 2: Now with Less Wrong Math — LessWrong window.__lwSsrGql.inject("query PostsPageWrapper($documentId: String, $sequenceId: String) {\n post(input: {selector: {documentId: $documentId}}, allowNull: true) {\n result {\n ...PostsWithNavigation\n }\n }\n}\n\nfragment PostsMinimumInfo on Post {\n _id\n slug\n title\n draft\n shortform\n hideCommentKarma\n af\n userId\n coauthorUserIds\n rejected\n collabEditorDialogue\n}\n\nfragment PostsBase on Post {\n ...PostsMinimumInfo\n url\n postedAt\n sticky\n metaSticky\n stickyPriority\n status\n frontpageDate\n meta\n deletedD

related reading

• Toward A Mathematical Framework for Computation in Superposition — LessWronglesswrong.com
• Toy Models of Superpositiontransformer-circuits.pub
• Ping pong computation in superposition — LessWronglesswrong.com
• Interpretability Dreamstransformer-circuits.pub
• Zoom In: An Introduction to Circuitsdistill.pub
• Computational Superposition in a Toy Model of the U-AND Problem — LessWronglesswrong.com
• Circuits Updates — May 2023transformer-circuits.pub
• Zoom In: An Introduction to Circuits — LessWronglesswrong.com
• What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
• Distributed Representations: Composition & Superpositiontransformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• On neural scaling and the quanta hypothesisericjmichaud.com