Chapter 1: Transformer Interpretability - ARENA

To emphasize - the idea is for this section is to be a whirlwind tour of all basic SAE topics, excluding training & evals (which we'll come back to in section 4). The focus will be on how to understand & interpret SAE latents (in particular all the components of the SAE dashboard). We'll also look at techniques for finding latents (e.g. ablation & attribution methods), as well as taking a deeper dive into attention SAEs and how they work. Because there's a lot of material to cover in this section, we'll have a summary of the key points at the top of each main header section. These summaries are all included below for convenience, before we get started. As well as helping to keep you oriented as you work through the material, these should also give you an idea of which sections you can jump to if you only want to cover a few of them. In this section, you'll learn what SAELens is, and how to use it to load in & inspect the configs of various supported SAEs. Key points: SAELens is a libra

1️⃣ Intro to SAE Interpretability Learning Objectives Learn how to use the SAELens library to load in & run SAEs (alongside the TransformerLens models they're attached to) Understand the basic features of Neuronpedia , and how it can be used for things like steering and searching over features Understand SAE dashboards , what each part of them tells you about a particular latent (as well as how to compute them yourself) Learn techniques for finding latents, including direct logit attribution , ablation and attribution patching Use attention SAEs , understand how they differ from regular SAEs (

related reading

• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• pdfopenreview.net
• Transformer Circuits Threadtransformer-circuits.pub
• Neuronpedianeuronpedia.org
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2) — AI Alignment Forumalignmentforum.org
• Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2) — LessWronglesswrong.com
• Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2) — AI Alignment Forumalignmentforum.org