[2309.08600] Sparse Autoencoders Find Highly Interpretable Features in Language Models

Abstract:One of the roadblocks to a better understanding of neural networks' internals is \textit{polysemanticity}, where neurons appear to activate in multiple, semantically distinct contexts. Polysemanticity prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally. One hypothesised cause of polysemanticity is \textit{superposition}, where neural networks represent more features than they have neurons by assigning features to an overcomplete set of directions in activation space, rather than to individual neurons. Here, we attempt to identify those directions, using sparse autoencoders to reconstruct the internal activations of a language model. These autoencoders learn sets of sparsely activating features that are more interpretable and monosemantic than directions identified by alternative approaches, where interpretability is measured by automated methods. Moreover, we show that with our learned set of features, we can pinpoint the features that are causally responsible for counterfactual behaviour on the indirect object identification task \citep{wang2022interpretability} to a finer degree than previous decompositions. This work indicates that it is possible to resolve superposition in language models using a scalable, unsupervised method. Our method may serve as a foundation for future mechanistic interpretability work, which we hope will enable greater model transparency and steerability.

S PARSE AUTOENCODERS F IND H IGHLY I NTER - PRETABLE F EATURES IN L ANGUAGE M ODELS Hoagy Cunningham∗12 , Aidan Ewart∗13 , Logan Riggs∗1 , Robert Huben, Lee Sharkey4 1 EleutherAI, 2 MATS, 3 Bristol AI Safety Centre, 4 Apollo Research {hoagycunningham, aidanprattewart, logansmith5}@gmail.com A BSTRACT…

related reading

• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Interpreting Language Model Parametersgoodfire.ai
• Sparse Autoencoders Reveal Universal Feature Spaces Across Large Language Modelsarxiv.org
• Sparse Attention Post-Training for Mechanistic Interpretabilityarxiv.org