Computational Superposition in a Toy Model of the U-AND Problem — LessWrong

Thanks to @Linda Linsefors and @Kola Ayonrinde for reviewing the draft. …

x Computational Superposition in a Toy Model of the U-AND Problem — LessWrong Interpretability (ML & AI) Superposition AI Frontpage 18 Computational Superposition in a Toy Model of the U-AND Problem by Adam Newgas 27th Mar 2025 14 min read 2 18 Thanks to @Linda Linsefors and @Kola Ayonrinde for reviewing the draft. tl;dr: I built a toy model of the Universal-AND Problem described in Toward A Mathematical Framework for Computation in Superposition (CiS). It successfully learnt a solution to the problem, providing evidence that such computational superposition [1] can occur in the wild. In this

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Ping pong computation in superposition — LessWronglesswrong.com
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• [2608.27540] Towards a mathematical theory of superpositionarxiv.org
• Tiny TPUtinytpu.com
• Circuits in Superposition 2: Now with Less Wrong Math — LessWronglesswrong.com
• Reverse Engineering a Neural Network's Clever Solution to Binary Addition - Casey Primozic's Homepagecprimozic.net
• What if not Circuits? — LessWronglesswrong.com
• [2605.01192] Linear-Readout Floors and Threshold Recovery in Computation in Superpositionarxiv.org