I Trained a Language Model. Then I Built a Brain Scanner and Looked Inside It. | by Caleb DeLeeuw | Apr, 2026 | Medium

A DIY sparse autoencoder tutorial for DIY tiny language models : superposition hypothesis, feature visualization, and activation steering, on a free GPU you can use today. Most people who train a language model stop at the loss curve. The model learns, gets better at predicting text, and that’s it. But what does “better” actually mean inside the model? Which neurons fire when it reads a math problem? What’s happening in there when it sees the word “not”? What has it figured out that nobody explicitly taught it? I built nanochat-SAE to answer those questions. It’s an open-source toolkit that adds sparse autoencoder (SAE) training on top of Andrej Karpathy’s nanochat framework. The whole pipeline runs on a free Colab T4 in about six minutes. Google makes this resource free to the public with a google account. No cloud bill. No research lab affiliation required. (Set “T4" as the runtime in the top right corner in Colab.) One thing I’m proud of is that Andrej Karpathy (author of nanoGPT an

I Trained a Language Model. Then I Built a Brain Scanner and Looked Inside It. Caleb DeLeeuw 12 min read · Apr 15, 2026 -- Listen Share A DIY sparse autoencoder tutorial for DIY tiny language models : superposition hypothesis, feature visualization, and activation steering, on a free GPU you can use today. Press enter or click to view image in full size Most people who train a language model stop at the loss curve. The model learns, gets better at predicting text, and that’s it. But what does “better” actually mean inside the model? Which neurons fire when it reads a math problem? What’s happe

saved by

• N . Soma Sekhar

related reading

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• pdfopenreview.net
• Transformer Circuits Threadtransformer-circuits.pub
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• A gentle introduction to sparse autoencodersnickjiang.substack.com
• An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs — LessWronglesswrong.com
• Natural Language Autoencoders \ Anthropicanthropic.com
• Prism: mapping interpretable concepts and features in a latent space of language | thesephist.comthesephist.com
• [2309.08600] Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org