The Geometry of Concepts: Sparse Autoencoder Feature Structure

N/A

Article The Geometry of Concepts: Sparse Autoencoder Feature Structure Yuxiao Li 4,†, , Eric J. Michaud 1,3,† , David D. Baek 2,3,† , Joshua Engels 2 , Xiaoqing Sun 1 and Max Tegmark 1,3 * 1 Department of Physics, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA; ericjm@mit.edu (E.J.M.);…

related reading

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• [2602.15029] Symmetry in language statistics shapes the geometry of model representationsarxiv.org
• Prism: mapping interpretable concepts and features in a latent space of language | thesephist.comthesephist.com
• Sparse Autoencoders Reveal Universal Feature Spaces Across Large Language Modelsarxiv.org
• The World Inside Neural Networksgoodfire.ai
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizersgoodfire.ai