Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper — LessWrong

Anthropic recently put out a really cool paper about using Sparse Autoencoders (SAEs) to extract interpretable features from superposition in the MLP layer of a 1L language model. I think this is an awesome paper and I am excited about the potential of SAEs to be useful for mech interp more broadly, and to see how well they scale! This post documents a replication I did of their paper, along with some small explorations building on it, along with (scrappy!) training code and weights for some trained autoencoders. See an accompanying colab tutorial to load the trained autoencoders, and a demo of how to interpret what the features do. One of the things I find most interesting about this paper is the existence of non neuron basis aligned features. One of the big mysteries (by my lights) in mechanistic interpretability is what the non-linearities in MLP layers are actually doing, on an algorithmic level. I can reason about monosemantic GELU neurons fairly easily (like a French neuron) - es

x Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper — LessWrong Sparse Autoencoders (SAEs) AI Frontpage 93 Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper by Neel Nanda 23rd Oct 2023 AI Alignment Forum 11 min read 12 93 Ω 41 This is the long-form version of a public comment on Anthropic's Towards Monosemanticity paper Introduction Anthropic recently put out a really cool paper about using Sparse Autoencoders (SAEs) to extract interpretable features from superposition in the MLP layer of a 1L language model. I think this is an awesome pa

related reading

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• A gentle introduction to sparse autoencodersnickjiang.substack.com
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Toy Models of Superpositiontransformer-circuits.pub
• Do sparse autoencoders find "true features"? — LessWronglesswrong.com
• [Interim research report] Taking features out of superposition with sparse autoencoders — LessWronglesswrong.com
• pdfopenreview.net
• Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning — LessWronglesswrong.com
• Interpretability with Sparse Autoencoders (Colab exercises) — LessWronglesswrong.com