the j-lens: finding an llm's unspoken concepts / chirag

the theory and intuition behind anthropic's j-lens / j-space paper, with a little bit of maths.

this is part 1. theory and intuition, with a little bit of maths. paper: Verbalizable Representations Form a Global Workspace in Language Models let’s start with an example look at this: the prompt never says spider. the output is just 8. so the question is simple: can we detect the unspoken spider step, and show that the model actually uses it? the paper does both. it finds a spider-associated direction inside the residual stream, then swaps its coordinates for ant. the answer flips from 8 legs to 6. across 50 two-hop questions, this kind of swap succeeds in 54% of trials on Haiku 4.5…

saved by

• Ishaan Panigrahi

related reading

• A global workspace in language models \ Anthropicanthropic.com
• Jack Lindsey (@Jack_W_Lindsey) on Xx.com
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• What language do language models speak?tcz.hu
• Bridging the Attention Gap: Complete Replacement Models for Complete Circuit Tracinginterp.open-moss.com
• No Space Like J-Space - by Zvi Mowshowitzthezvi.substack.com
• The Annotated JEPA | Elements of a Vector Spaceelonlit.com
• A Review of Anthropic's Global Workspace Paper — LessWronglesswrong.com
• Understanding the J-Lensemma-x1.github.io
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• On the Biology of a Large Language Modeltransformer-circuits.pub