Sparse Autoencoders Find Highly Interpretable Features in Language Models

N/A

# link_2co6mhcts99.pdf ## Metadata - PDFFormatVersion=1.5 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - CreationDate=D:20231005020919Z - Creator=LaTeX with hyperref - ModDate=D:20231005020919Z - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) kpathsea version 6.3.5 - Producer=pdfTeX-1.40.25 - Trapped=False ## Contents ### Page 1 SPARSE AUTOENCODERS FIND HIGHLY INTER-PRETABLE FEATURES IN LANGUAGE MODELSHoagy Cunningham∗12, Aidan Ewart∗13, Logan Riggs∗1, Robert Huben, Lee Sha

saved by

• Michael Li
• Sidhant Chaliha

related reading

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• GitHub - PaulPauls/llama3_interpretability_sae: A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.github.com
• [2309.08600] Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• Toy Models of Superpositiontransformer-circuits.pub
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• pdfopenreview.net
• A gentle introduction to sparse autoencodersnickjiang.substack.com
• Anthropic Dictionary Learning Public Commentdocs.google.com
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub