Transluce on X: "Is your LM secretly an SAE? Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits! https://t.co/lTBbUqoRlt" / Twitter

To view keyboard shortcuts, press question mark View keyboard shortcuts Added to your Bookmarks Add to Folder Tweet See new posts Conversation Neil Rathi reposted Transluce @TransluceAI Is your LM secretly an SAE? Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits! 11:00 AM · Nov 20, 2025 ·Twitter Web App 9 Quote Tweets 5 74 249 216 Tweet your reply Reply Transluce @TransluceAI · Nov 20 Comparing with sparse autoencoders (SAEs), we show that circuits traced directly on a model’s MLP neurons can be just as sparse and faithful. We use two advances to achieve this result. 1 22 Transluce @TransluceAI · Nov 20 First, we use MLP neurons instead of MLP outputs as the units of our circuits. Due to the preceding nonlinearity, features are encouraged to align with the MLP neuron basis. 1 21 Transl

Transluce @TransluceAI Is your LM secretly an SAE? Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits! 7:00 PM · Nov 20, 2025 133.2K Views 7 0 7 77 0 7 7 376 0 3 7 6 345 0 3 4 5 Read 7 replies

related reading

• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• [2406.11944] Transcoders Find Interpretable LLM Feature Circuitsarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Circuit Tracing in Vision–Language Models:Understanding the Internal Mechanisms of Multimodal Thinkingarxiv.org
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Towards Automated Circuit Discovery for Mechanistic Interpretabilityarxiv.org