Circuits Updates - July 2024

We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather than a mature paper. New Posts If you'd asked me a year ago what the key open problems for mechanistic interpretability were, I would have told you the most important problem was superposition.Since then, very significant progress has been made on superposition (e.g. Bricken et al., Cunningham et al., Templeton et al., Gao et al.). As a result, it feels like a good moment to take stock and ask "what are the remaining hurdles between us and having a mechanistic understanding of neural net

Circuits Updates - July 2024 Transformer Circuits Thread Circuits Updates - July 2024 We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them. We'd ask you to treat these results like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather than a

related reading

• Circuits Updates — May 2023transformer-circuits.pub
• Circuits Updates - April 2024transformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Interpretability Dreamstransformer-circuits.pub
• Circuits Updates - January 2024transformer-circuits.pub
• Circuits Updates - April 2025transformer-circuits.pub
• On the Biology of a Large Language Modeltransformer-circuits.pub
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Toy Models of Superpositiontransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub