Understanding Transformers... (beyond the Math) | kalomaze's kalomazing blog
Author's Note This is an experimental piece - I recorded an hour long voice memo while on a walk, talking to myself out loud, stream of consciousness style....
Understanding Transformers... (beyond the Math) – kalomaze's kalomazing blog Understanding Transformers... (beyond the Math) 09 Mar, 2025 Author's Note This is an experimental piece - I recorded an hour long voice memo while on a walk, talking to myself out loud, stream of consciousness style. Some things are probably worded quite messily, but it's 3 times longer compared to what I've written before, and I've found that cleaning up the transcriptions is a significantly faster process compared to writing from scratch (what I've done here took me less than ~4 hours total) I figure that this styl
Explore this link on the map →related reading
- Transformer Circuits Threadtransformer-circuits.pub
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
- In-context Learning and Induction Headstransformer-circuits.pub
- Transformers from Scratche2eml.school
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
- Transformers Represent Belief State Geometry in their Residual Stream — LessWronglesswrong.com
- GenAI Handbookgenai-handbook.github.io
- The Annotated Transformernlp.seas.harvard.edu
- The Topological Trouble With Transformersarxiv.org
- Extending Context is Hard | kaiokendevkaiokendev.github.io