On the Biology of a Large Language Model

We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model — in a variety of contexts, using our circuit tracing methodology.

On the Biology of a Large Language Model × Transformer Circuits Thread On the Biology of a Large Language Model On the Biology of a Large Language Model We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model — in a variety of contexts, using our circuit tracing methodology. Authors Jack Lindsey † , Wes Gurnee * , Emmanuel Ameisen * , Brian Chen * , Adam Pearce * , Nicholas L. Turner * , Craig Citro * , David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall ◊ , Hoagy Cunning

saved by

• Winnie Xu
• Trang Đoàn
• abinaya dinesh
• Tasha Pais
• Ratan Kaliani
• Freeman Jiang
• Aryan Naik
• Samuel Lo
• Matthew Wang
• Asher P
• Vyom Pathak
• Lydia Nottingham

related reading

• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• In-context Learning and Induction Headstransformer-circuits.pub
• A global workspace in language models \ Anthropicanthropic.com
• Emotion Concepts and their Function in a Large Language Modeltransformer-circuits.pub
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Mapping the Mind of a Large Language Model \ Anthropicanthropic.com
• Tracing the Thoughts of a Large Language Model — LessWronglesswrong.com
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• [2210.13382] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskarxiv.org