Circuits Updates - February 2024

We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather than a mature paper. Short Research Notes In Towards Monosemanticity, we reported that, for very wide autoencoders, most features ended up in an ‘ultralow density cluster’ of the feature density histogram. These features fire very rarely, are not obviously interpretable, and take up a majority of autoencoder capacity, creating a barrier to scaling up autoencoders to a large number of features. A replication by Neel Nanda pointed out that the encoder vectors for these features had extreme

Circuits Updates - February 2024 Transformer Circuits Thread Circuits Updates - February 2024 We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rathe

related reading

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• [Interim research report] Taking features out of superposition with sparse autoencoders — LessWronglesswrong.com
• Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper — LessWronglesswrong.com
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Circuits Updates - January 2024transformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Improving Dictionary Learning with Gated Sparse Autoencodersarxiv.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com