Addressing Feature Suppression in SAEs — LessWrong

Produced as part of the ML Alignment Theory Scholars Program - Winter 2023-24 Cohort as part of Lee Sharkey's stream. …

x Addressing Feature Suppression in SAEs — LessWrong Sparse Autoencoders (SAEs) Interpretability (ML & AI) AI Frontpage 88 Addressing Feature Suppression in SAEs by Benjamin Wright , Lee Sharkey 16th Feb 2024 AI Alignment Forum 12 min read 5 88 Ω 46 Produced as part of the ML Alignment Theory Scholars Program - Winter 2023-24 Cohort as part of Lee Sharkey's stream. TL;DR Sparse autoencoders are a method of resolving superposition by recovering linearly encoded “features” inside activations. Unfortunately, despite the great recent success of SAEs at extracting human interpretable features, they

related reading

• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• Matryoshka Sparse Autoencoders — LessWronglesswrong.com
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning — LessWronglesswrong.com
• Do sparse autoencoders find "true features"? — LessWronglesswrong.com
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• An Introduction to Exemplar Partitioning for Mechanistic Interpretability — LessWronglesswrong.com
• An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs — LessWronglesswrong.com
• pdfopenreview.net
• Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs — LessWronglesswrong.com