Gears-Level Mental Models of Transformer Interpretability — LessWrong

This post aims to quickly break down and explain the dominant mental models interpretability researchers currently use when thinking about how transf…

x Gears-Level Mental Models of Transformer Interpretability — LessWrong Interpretability (ML & AI) Language Models (LLMs) AI Frontpage 77 Gears-Level Mental Models of Transformer Interpretability by RowanWang 29th Mar 2022 AI Alignment Forum 7 min read 4 77 Ω 25 This post aims to quickly break down and explain the dominant mental models interpretability researchers currently use when thinking about how transformers work. In my view, the focus of transformer interpretability research is teleological: we care about the functions each component in the transformer performs and how those functions

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• interpreting GPT: the logit lens — LessWronglesswrong.com
• An Analogy for Understanding Transformers — LessWronglesswrong.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• In-context Learning and Induction Headstransformer-circuits.pub
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com
• Thinking like Transformersrush.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Transformer Inference Arithmetic | kipply's blogkipp.ly