What I learned this week - Pretraining parallelisms, Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RL, On why pretraining runs fails

April 15, 2025

Blog What I learned this week - Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RL April 15, 2025 Dwarkesh Patel Apr 15, 2026 133 16 6 Share At the end of my conversation with Michael Nielsen , we talked about how to actually retain what you learn. Michael’s advice was to make some kind of demanding artifact. Write something up. Try to explain it. So in that spirit, here are notes on some topics I’ve learned about over the last week or two. These notes are extremely rough, and have many mistakes. Can distillation be stopped? Can the frontier labs stop distillati

saved by

• Abhay Sheshadri
• Ishan Mukherjee
• Will Anderson
• Ishaan Panigrahi

related reading

• Alignment is not solved but it increasingly looks solvablealigned.substack.com
• Why Frontier AI Labs Fight to Hide Chain of Thought — Ilia Shumailov & Alexander Panfilovyoutube.com
• Thoughts on AI progress (Dec 2025)substack.com
• AI researchers debate how close we are to recursive self-improvementyoutube.com
• On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
• AI researchers talk continual learning over a game of HORSEyoutube.com
• Let’s talk about encrypted reasoningblog.cryptographyengineering.com
• RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com
• You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
• Don’t Outsource Your Thinkingteltam.github.io
• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io