An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonen

Sparse Autoencoders (SAEs) have recently become popular for interpretability of machine learning models (although sparse dictionary learning has been around since 1997). Machine learning models and LLMs are becoming more powerful and useful, but they are still black boxes, and we don’t understand how they do the things that they are capable of. It seems like it would be useful if we could understand how they work.

Sparse Autoencoders (SAEs) have recently become popular for interpretability of machine learning models (although sparse dictionary learning has been around since 1997 ). Machine learning models and LLMs are becoming more powerful and useful, but they are still black boxes, and we don’t understand how they do the things that they are capable of. It seems like it would be useful if we could understand how they work. Using SAEs, we can begin to break down a model’s computation into understandable components . There are several existing explanations of SAEs, and I wanted to create a brief writeup

saved by

• jeremy h
• Yoyo Yuan
• Lydia Nottingham
• Vincent Cheng
• Timothy Kostolansky
• Adi Mehta
• Jackson Mowatt Gok
• Ishaan Panigrahi

related reading

• An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs — LessWronglesswrong.com
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• A practical guide to sparse autoencoders (SAEs) - Goodfiregoodfire.com
• A gentle introduction to sparse autoencodersnickjiang.substack.com
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Faithful, Interpretable Model Explanations via Causal Abstraction | SAIL Blogai.stanford.edu
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• GitHub - PaulPauls/llama3_interpretability_sae: A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.github.com
• SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilityarxiv.org