I Trained a Language Model. Then I Built a Brain Scanner and Looked Inside It. | by Caleb DeLeeuw | Apr, 2026 | Medium

A DIY sparse autoencoder tutorial for DIY tiny language models : superposition hypothesis, feature visualization, and activation steering, on a free GPU you can use today. Most people who train a language model stop at the loss curve. The model learns, gets better at predicting text, and that’s it. But what does “better” actually mean inside the model? Which neurons fire when it reads a math problem? What’s happening in there when it sees the word “not”? What has it figured out that nobody explicitly taught it? I built nanochat-SAE to answer those questions. It’s an open-source toolkit that adds sparse autoencoder (SAE) training on top of Andrej Karpathy’s nanochat framework. The whole pipeline runs on a free Colab T4 in about six minutes. Google makes this resource free to the public with a google account. No cloud bill. No research lab affiliation required. (Set “T4" as the runtime in the top right corner in Colab.) One thing I’m proud of is that Andrej Karpathy (author of nanoGPT an

I Trained a Language Model. Then I Built a Brain Scanner and Looked Inside It. Caleb DeLeeuw 12 min read · Apr 15, 2026 -- Listen Share A DIY sparse autoencoder tutorial for DIY tiny language models : superposition hypothesis, feature visualization, and activation steering, on a free GPU you can use today. Press enter or click to view image in full size Most people who train a language model stop at the loss curve. The model learns, gets better at predicting text, and that’s it. But what does “better” actually mean inside the model? Which neurons fire when it reads a math problem? What’s happe

saved by

• N . Soma Sekhar

related reading

• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Transformer Circuits Threadtransformer-circuits.pub
• pdfopenreview.net
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• A gentle introduction to sparse autoencodersnickjiang.substack.com
• Llama Scope: Extracting Features from Llama 3.1-8B with SAEsarxiv.org
• An Intuitive Explanation of Sparse Autoencoders for Mechanistic Interpretability of LLMs — LessWronglesswrong.com