interpreting GPT: the logit lens - AI Alignment Forum

This post relates an observation I've made in my work with GPT-2, which I have not seen made elsewhere. …

x interpreting GPT: the logit lens — AI Alignment Forum GPT Machine Learning (ML) Gears-Level Interpretability (ML & AI) AI Frontpage 80 interpreting GPT: the logit lens by nostalgebraist 31st Aug 2020 13 min read 38 80 This post relates an observation I've made in my work with GPT-2, which I have not seen made elsewhere. IMO, this observation sheds a good deal of light on how the GPT-2/3/etc models (hereafter just "GPT") work internally. There is an accompanying Colab notebook which will let you interactively explore the phenomenon I describe here. [Edit: updated with another section on compa

related reading

• interpreting GPT: the logit lens — LessWronglesswrong.com
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Simulators — LessWronglesswrong.com
• GPT in 60 Lines of NumPy | Jay Modyjaykmody.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
• Let's build GPT: from scratch, in code, spelled out.youtube.com
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Thoughts on the Alignment Implications of Scaling Language Models | Leo Gaobmk.sh
• microgptkarpathy.github.io
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com
• The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable — LessWronglesswrong.com