A gentle introduction to sparse autoencoders

Our biggest leap so far in deciphering how large language models work

A gentle introduction to sparse autoencoders Our biggest leap so far in deciphering how large language models work Nick Jiang Jul 03, 2024 6 Share DALLE3: a lab technician wielding a sledgehammer to break open a black box Sparse autoencoders (SAEs) are the current hot topic 🔥 in the interpretability world. In late May, Anthropic released a paper that shows how to use sparse autoencoders to effectively break down the internal reasoning of Claude 3 (Anthropic’s LLM) 1 . Shortly after, OpenAI published a paper successfully applying a similar procedure for GPT4. What’s exciting about SAEs is that

related reading

• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs — LessWronglesswrong.com
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• GitHub - PaulPauls/llama3_interpretability_sae: A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.github.com
• A practical guide to sparse autoencoders (SAEs) - Goodfiregoodfire.com
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• An Introduction to Exemplar Partitioning for Mechanistic Interpretability — LessWronglesswrong.com
• Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper — LessWronglesswrong.com
• Matryoshka Sparse Autoencoders — LessWronglesswrong.com