the j-lens: finding an llm's unspoken concepts / chirag
ctxnn.github.io · 2,225 words · saved by 1 readers
the theory and intuition behind anthropic's j-lens / j-space paper, with a little bit of maths.
this is part 1. theory and intuition, with a little bit of maths. paper: Verbalizable Representations Form a Global Workspace in Language Models let’s start with an example look at this: the prompt never says spider. the output is just 8. so the question is simple: can we detect the unspoken spider step, and show that the model actually uses it? the paper does both. it finds a spider-associated direction inside the residual stream, then swaps its coordinates for ant. the answer flips from 8 legs to 6. across 50 two-hop questions, this kind of swap succeeds in 54% of trials on Haiku 4.5…
saved by
related reading
- Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
- A global workspace in language models \ Anthropicanthropic.com
- What language do language models speak?tcz.hu
- How LLMs Actually Work | 0xkato0xkato.xyz
- A Review of Anthropic's Global Workspace Paper — LessWronglesswrong.com
- No Space Like J-Space - by Zvi Mowshowitzthezvi.substack.com
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- Transformer Circuits Threadtransformer-circuits.pub
- Understanding the J-Lensemma-x1.github.io
- Jacobian Lens – Qwen3.6-27B ｜ Neuronpedianeuronpedia.org
- Tracing the thoughts of a large language model \ Anthropicanthropic.com
- interpreting GPT: the logit lens — LessWronglesswrong.com