Tracing Attention Computation Through Feature Interactions

Transformer-based language models involve two main kinds of computations: multi-layer perceptron (MLP) layers that process information within a context position, and attention layers that conditionally move and process information between context positions. In our recent papers we made significant progress in breaking down MLP computation into interpretable steps. In this update, we fill in a major missing piece in our methodology, by introducing a way to decompose attentional computations as well.

Tracing Attention Computation Through Feature Interactions Transformer Circuits Thread Tracing Attention Computation Through Feature Interactions We describe and apply a method to explain attention patterns in terms of feature interactions, and integrate this information into attribution graphs. Authors Harish Kamath * , Emmanuel Ameisen * , Isaac Kauvar, Rodrigo Luger, Wes Gurnee, Adam Pearce, Sam Zimmerman, Joshua Batson, Thomas Conerly, Chris Olah, Jack Lindsey ‡ Affiliations Anthropic Published July 31st, 2025 * Core Research Contributor; ‡ Correspondence to jacklindsey@anthropic.com Trans

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Transformer Circuits Threadtransformer-circuits.pub
• On the Biology of a Large Language Modeltransformer-circuits.pub
• transformer_attention.pdfarxiv.org
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Transformers from Scratche2eml.school
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• 1706.03762arxiv.org
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com