Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning — LessWrong

A short summary of the paper is presented below. …

x Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning — LessWrong Apollo Research (org) Interpretability (ML & AI) Sparse Autoencoders (SAEs) AI Frontpage 57 Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning by Dan Braun , Jordan Taylor , Nicholas Goldowsky-Dill , Lee Sharkey 17th May 2024 AI Alignment Forum Linkpost for arxiv.org 5 min read 20 57 Ω 28 A short summary of the paper is presented below. This work was produced by Apollo Research in collaboration with Jordan Taylor (MATS + University of Queensland) . TL;DR: We

saved by

• Celeste

related reading

• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• An Introduction to Exemplar Partitioning for Mechanistic Interpretability — LessWronglesswrong.com
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Matryoshka Sparse Autoencoders — LessWronglesswrong.com
• Do sparse autoencoders find "true features"? — LessWronglesswrong.com
• Addressing Feature Suppression in SAEs — LessWronglesswrong.com
• Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper — LessWronglesswrong.com
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• Improving Dictionary Learning with Gated Sparse Autoencodersarxiv.org
• Efficient Dictionary Learning with Switch Sparse Autoencoders — LessWronglesswrong.com