✳flâneur — a map of the web's best reading
Transformers Represent Belief State Geometry in their Residual Stream — LessWrong
lesswrong.com · 14,324 words · saved by 1 readers
Produced while being an affiliate at PIBBSS[1]. The work was done initially with funding from a Lightspeed Grant, and then continued while at PIBBSS.…
x Transformers Represent Belief State Geometry in their Residual Stream — LessWrong Best of LessWrong 2024 MATS Program PIBBSS Interpretability (ML & AI) Transformers AI Curated 442 Transformers Represent Belief State Geometry in their Residual Stream by Adam Shai 16th Apr 2024 AI Alignment Forum 14 min read 103 442 Ω 145 Produced while being an affiliate at PIBBSS [1] . The work was done initially with funding from a Lightspeed Grant, and then continued while at PIBBSS. Work done in collaboration with @Paul Riechers , @Lucas Teixeira , @Alexander Gietelink Oldenziel , and Sarah Marzen . Paul
Explore this link on the map →related reading
- Simplex Progress Report - July 2025 — LessWronglesswrong.com
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- Transformer Circuits Threadtransformer-circuits.pub
- The Topological Trouble With Transformersarxiv.org
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
- Transformers from Scratche2eml.school
- Understanding Transformers... (beyond the Math) – kalomaze's kalomazing blogkalomaze.bearblog.dev
- A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
- Gears-Level Mental Models of Transformer Interpretability — LessWronglesswrong.com
- An Analogy for Understanding Transformers — LessWronglesswrong.com
- GenAI Handbookgenai-handbook.github.io