TransformerLens Documentation

This is a library for doing mechanistic interpretability of GPT-2 Style language models. The goal of mechanistic interpretability is to take a trained model and reverse engineer the algorithms the model learned during training from its weights. It is a fact about the world today that we have computer programs that can essentially speak English at a human level (GPT-3, PaLM, etc), yet we have no idea how they work nor how to write one ourselves. This offends me greatly, and I would like to solve this! TransformerLens lets you load in an open source language model, like GPT-2, and exposes the internal activations of the model to you. You can cache any internal activation in the model, and add in functions to edit, remove or replace these activations as the model runs. The core design principle I’ve followed is to enable exploratory analysis. One of the most fun parts of mechanistic interpretability compared to normal ML is the extremely short feedback loops! The point of this library is

TransformerLens ¶ (Formerly known as EasyTransformer) A Library for Mechanistic Interpretability of Generative Language Models ¶ This is a library for doing mechanistic interpretability of GPT-2 Style language models. The goal of mechanistic interpretability is to take a trained model and reverse engineer the algorithms the model learned during training from its weights. It is a fact about the world today that we have computer programs that can essentially speak English at a human level (GPT-3, PaLM, etc), yet we have no idea how they work nor how to write one ourselves. This offends me greatl

related reading

• TransformerLens Documentationtransformerlensorg.github.io
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Transformer Circuits Threadtransformer-circuits.pub
• In-context Learning and Induction Headstransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Exploratory Analysis Demo - TransformerLens Documentationtransformerlensorg.github.io
• interpreting GPT: the logit lens — LessWronglesswrong.com
• Thinking like Transformersrush.github.io
• Transformers from Scratche2eml.school
• A Comprehensive Mechanistic Interpretability Explainer & Glossary — Neel Nandaneelnanda.io