An Analogy for Understanding Transformers — LessWrong

Thanks to the following people for feedback: Tilman Rauker, Curt Tigges, Rudolf Laine, Logan Smith, Arthur Conmy, Joseph Bloom, Rusheb Shah, James Dao. I present an analogy for the transformer architecture: each vector in the residual stream is a person standing in a line, who is holding a token, and trying to guess what token the person in front of them is holding. Attention heads represent questions that people in this line can ask to everyone standing behind them (queries are the questions, keys determine who answers the questions, values determine what information gets passed back to the original question-asker), and MLPs represent the internal processing done by each person in the line. I claim this is a useful way to intuitively understand the transformer architecture, and I'll present several reasons for this (as well as ways induction heads and indirect object identification can be understood in these terms).[1] In this post, I'm going to present an analogy for understanding ho

x An Analogy for Understanding Transformers — LessWrong Transformer Circuits Transformers AI Frontpage 92 An Analogy for Understanding Transformers by CallumMcDougall 13th May 2023 11 min read 6 92 Thanks to the following people for feedback: Tilman Rauker, Curt Tigges, Rudolf Laine, Logan Smith, Arthur Conmy, Joseph Bloom, Rusheb Shah, James Dao. TL;DR I present an analogy for the transformer architecture: each vector in the residual stream is a person standing in a line, who is holding a token, and trying to guess what token the person in front of them is holding. Attention heads represent q

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Gears-Level Mental Models of Transformer Interpretability — LessWronglesswrong.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Transformers from Scratche2eml.school
• Transformer Circuits Threadtransformer-circuits.pub
• Everything About Transformerskrupadave.com
• The Annotated Transformernlp.seas.harvard.edu
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Some Intuition on Attention and the Transformereugeneyan.com
• 9 Transformers – 6.390 - Intro to Machine Learningintroml.mit.edu