Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWrong

TLDR: Recently, Gao et al trained transformers with sparse weights, and introduced a pruning algorithm to extract circuits that explain performance o…

x Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWrong Interpretability (ML & AI) AI Frontpage 2026 Top Fifty: 9 % 136 Weight-Sparse Circuits May Be Interpretable Yet Unfaithful by jacob_drori 9th Feb 2026 9 min read 5 136 TLDR: Recently, Gao et al trained transformers with sparse weights, and introduced a pruning algorithm to extract circuits that explain performance on narrow tasks. I replicate their main results and present evidence suggesting that these circuits are unfaithful to the model’s “true computations”. This work was done as part of the Anthropic Fellows Program

saved by

• Asher P
• Yudhister Joel Kumar
• Will Anderson

related reading

• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Insights on Crosscoder Model Diffingtransformer-circuits.pub
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Matryoshka Sparse Autoencoders — LessWronglesswrong.com
• Attribution-based parameter decomposition — LessWronglesswrong.com
• Sparse Attention Post-Training for Mechanistic Interpretabilityarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Bridging the Attention Gap: Complete Replacement Models for Complete Circuit Tracinginterp.open-moss.com