Ping pong computation in superposition — LessWrong

Overview: This post builds on Circuits in Superposition 2, using the same terminology. …

x Ping pong computation in superposition — LessWrong Comp-In-Sup Interpretability (ML & AI) Superposition AI Frontpage 13 Ping pong computation in superposition by Alex Gibson 29th Dec 2025 4 min read 0 13 Overview: This post builds on Circuits in Superposition 2 , using the same terminology. I focus on the z = 1 case, meaning that exactly one circuit is active on each forward pass. This restriction simplifies the setting substantially and allows a construction with zero error, with T = D 2 d 2 , L + 2 layers, and width D ( 1 + 1 d + ⌈ log 2 D d ⌉ D ) . While the construction does not directly

saved by

• Asher P

related reading

• Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
• AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
• Tensor-Transformer Variants are Surprisingly Performant — LessWronglesswrong.com
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Toy Models of Superpositiontransformer-circuits.pub
• Circuits in Superposition 2: Now with Less Wrong Math — LessWronglesswrong.com
• Computational Superposition in a Toy Model of the U-AND Problem — LessWronglesswrong.com
• Tiny TPUtinytpu.com
• [2608.27540] Towards a mathematical theory of superpositionarxiv.org
• [2605.01192] Linear-Readout Floors and Threshold Recovery in Computation in Superpositionarxiv.org
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io