✳flâneur — a map of the web's best reading
Pre, Mid, Post-Training Way of Life - by Tina He
fakepixels.substack.com · 3,070 words · saved by 5 readers
Clarity has very little to do with the engine of the intellect.
Pre, Mid, Post-Training Way of Life Clarity has very little to do with the engine of the intellect. Tina He Dec 26, 2025 66 15 Share "For here there is no place / that does not see you. You must change your life." — Rilke’s Archaic Torso of Apollo Building a large language model happens in three main stages . Pre-training processes trillions of tokens scraped from everywhere, taken in with little discrimination. The model learns to predict the next word from the last, and the deceptively simple task demands massive compute. Tens of thousands of GPUs for months, tolerating messy, uncurated data
Explore this link on the map →saved by
related reading
- Dario Amodei — "We are near the end of the exponential"dwarkesh.com
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- llm assistant personas seem increasingly incoherent (some subjective observations) — LessWronglesswrong.com
- Elicitation, the simplest way to understand post-traininginterconnects.ai
- [2606.12360] Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signalarxiv.org
- Feedbackloop-first Rationality — LessWronglesswrong.com
- MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
- Modern Pretraining Strategies: A Hands-On Guidetheneuralmaze.substack.com
- How far does alignment midtraining generalize?alignment.openai.com
- RL is even more information inefficient than you thoughtdwarkesh.com
- Generalization Dynamics of LM Pre-training — Jiaxin Wenjiaxin-wen.github.io
- Rationality Research Report: Towards 10x OODA Looping? — LessWronglesswrong.com