What if not Circuits? — LessWrong

This post was written as part of the Iliad Fellowship. Inspired by conversations with Richard Ngo, Dmitry Vaintrob, and Brianna Grado-White. To all o…

This post was written as part of the Iliad Fellowship. Inspired by conversations with Richard Ngo, Dmitry Vaintrob, and Brianna Grado-White. To all of these, my thanks. Preface: I'm confused about how neural networks do and learn computations. In response to a friend's challenge, I'm writing up some interim thoughts. This essay has four parts: the first tries to track what I call the 'default ontology' of the mechinterp community over the years. The second part is about 'representational drift' as an important obstacle to weights-based approaches to circuits. The third part reflects on how…

saved by

• John Mathena
• Ishaan Panigrahi
• Evie Hu

related reading

• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyartyoutube.com
• What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
• Strange Geometric Shapes Found Inside AIs — Tom McGrathyoutube.com
• Transformer Circuits Threadtransformer-circuits.pub
• A short note on interpretability and mindsericjmichaud.com
• Neural networks are not already cleanlesswrong.com
• Zoom In: An Introduction to Circuitsdistill.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• The Iliad Intensive Course Materials — LessWronglesswrong.com
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• The World Inside Neural Networksgoodfire.ai