A Mathematical Framework for Transformer Circuits

Transformer language models are an emerging technology that is gaining increasingly broad real-world use, for example in systems like GPT-3 , LaMDA , Codex , Meena , Gopher , and similar models.  However, as these models scale, their open-endedness and high capacity creates an increasing scope for unexpected and sometimes harmful behaviors.  Even years after a large model is trained, both creators and users routinely discover model capabilities – including problematic behaviors – they were previously unaware of.

A Mathematical Framework for Transformer Circuits Transformer Circuits Thread A Mathematical Framework for Transformer Circuits Authors Nelson Elhage ∗† , Neel Nanda ∗ , Catherine Olsson ∗ , Tom Henighan † , Nicholas Joseph † , Ben Mann † , Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds , Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, Chris Olah ‡ Affiliation Anthropic Published Dec 22, 2021 * Core Research Contributor; † Core Infrastruc

saved by

• Winnie Xu
• Alex K. Chen
• Klaudia U.
• c l
• Pier-Luc Nadeau-Labadie
• Michelle Pan
• Claire Wang
• Asma Lamgh
• Alex Shaw
• Shubham Chandel
• Ivy Zhang
• James Webb

related reading

• In-context Learning and Induction Headstransformer-circuits.pub
• On the Biology of a Large Language Modeltransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Softmax Linear Unitstransformer-circuits.pub
• transformer_attention.pdfarxiv.org
• Transformers from Scratche2eml.school
• Towards Automated Circuit Discovery for Mechanistic Interpretabilityarxiv.org
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Thinking like Transformersrush.github.io
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com