TransformerLens Documentation

This is a library for doing mechanistic interpretability of GPT-2 Style language models. The goal of mechanistic interpretability is to take a trained model and reverse engineer the algorithms the model learned during training from its weights. It is a fact about the world today that we have computer programs that can essentially speak English at a human level (GPT-3, PaLM, etc), yet we have no idea how they work nor how to write one ourselves. This offends me greatly, and I would like to solve this! TransformerLens lets you load in an open source language model, like GPT-2, and exposes the internal activations of the model to you. You can cache any internal activation in the model, and add in functions to edit, remove or replace these activations as the model runs. The core design principle I’ve followed is to enable exploratory analysis. One of the most fun parts of mechanistic interpretability compared to normal ML is the extremely short feedback loops! The point of this library is

TransformerLens ¶ (Formerly known as EasyTransformer) A Library for Mechanistic Interpretability of Generative Language Models ¶ This is a library for doing mechanistic interpretability of GPT-2 Style language models. The goal of mechanistic interpretability is to take a trained model and reverse engineer the algorithms the model learned during training from its weights. It is a fact about the world today that we have computer programs that can essentially speak English at a human level (GPT-3, PaLM, etc), yet we have no idea how they work nor how to write one ourselves. This offends me greatl

related reading

• TransformerLens Documentationtransformerlensorg.github.io
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Transformer Circuits Threadtransformer-circuits.pub
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Language models can explain neurons in language modelsopenaipublic.blob.core.windows.net
• How To Become A Mechanistic Interpretability Researcher — AI Alignment Forumalignmentforum.org
• Prism: mapping interpretable concepts and features in a latent space of language | thesephist.comthesephist.com
• Neuronpedianeuronpedia.org