Interpretability Dreams

An informal note on the relationship between superposition and distributed representations by Chris Olah. Published May 24th, 2023. Not published yet. No DOI yet. Our present research aims to create a foundation for mechanistic interpretability research. In particular, we're focused on trying to resolve the challenge of superposition. In doing so, it's important to keep sight of what we're trying to lay the foundations for. This essay summarizes those motivating aspirations – the exciting directions we hope will be possible if we can overcome the present challenges. We aim to offer insight into our vision for addressing mechanistic interpretability's other challenges, especially scalability. Because we have focused on foundational issues, our longer-term path to scaling interpretability and tackling other challenges has often been obscure. By articulating this vision, we hope to clarify how we might resolve limitations, like analyzing massive neural networks, that might naively seem in

Interpretability Dreams Transformer Circuits Thread Interpretability Dreams AKA The Vistas Beyond Superposition // Beyond the Foundations of Mechanistic Interpretability // … --> An informal note on future goals for mechanistic interpretability by Chris Olah. Published May 24th, 2023. Our present research aims to create a foundation for mechanistic interpretability research. In particular, we're focused on trying to resolve the challenge of superposition . In doing so, it's important to keep sight of what we're trying to lay the foundations for. This essay summarizes those motivating aspiratio

saved by

• Asma Lamgh
• Ivy Zhang
• Shreya Mogulothu
• Vihaan Sondhi

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Circuits Updates — May 2023transformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Circuits Updates - July 2024transformer-circuits.pub
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• The Building Blocks of Interpretabilitydistill.pub
• What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Circuits Updates - January 2024transformer-circuits.pub
• Distributed Representations: Composition & Superpositiontransformer-circuits.pub