When Models Manipulate Manifolds: The Geometry of a Counting Task

Intelligent systems need perception to understand, predict, and navigate their environment. These sensory capabilities reflect what's useful for survival in a specific environment: bats use echolocation, migratory birds sense magnetic fields, Arctic reindeer shift their UV vision seasonally. But when your world is made of text, what do you see?  Language models encounter many text-based tasks that benefit from visual or spatial reasoning: parsing ASCII art, interpreting tables, or handling text wrapping constraints. Yet their only “sensory” input is a sequence of integers representing tokens. They must learn perceptual abilities from scratch, developing specialized mechanisms in the process. In this work, we investigate the mechanisms that enable Claude 3.5 Haiku to perform a natural perceptual task which is common in pretraining corpora and involves tracking position in a document. We find learned representations of position that are in some ways quite similar to the biological neuron

When Models Manipulate Manifolds: The Geometry of a Counting Task Transformer Circuits Thread When Models Manipulate Manifolds: The Geometry of a Counting Task When Models Manipulate Manifolds: The Geometry of a Counting Task Authors Wes Gurnee * , Emmanuel Ameisen * , Isaac Kauvar, Julius Tarng, Adam Pearce, Chris Olah, Joshua Batson *‡ Affiliations Anthropic Published October 21st, 2025 * Core Research Contributor; ‡ Correspondence to joshb@anthropic.com Introduction Intelligent systems need perception to understand, predict, and navigate their environment. These sensory capabilities reflect

saved by

• Rishi Kothari
• Lydia Nottingham
• Sudarsh K
• Logan Graves
• Cheikh Fiteni
• Emil Ryd
• Jackson Mowatt Gok
• Riya Tyagi
• Sam B

related reading

• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• On the Biology of a Large Language Modeltransformer-circuits.pub
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizersgoodfire.ai
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• [2210.13382] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• The World Inside Neural Networksgoodfire.ai
• In-context Learning and Induction Headstransformer-circuits.pub