Glenn Matlin on X: "🤔Where in trillions of pre-training tokens do capabilities actually come from? 💥In our new COLM 2026 paper, we trace the capability provenance of OLMo3-7B across Dolma3 using gradient-based training data attribution 🧵 Paper: https://t.co/rqXOeJAshH Code: https://t.co/sdaJ32iHSB" / X
🤔Where in trillions of pre-training tokens do capabilities actually come from? 💥In our new COLM 2026 paper, we trace the capability provenance of OLMo3-7B across Dolma3 using gradient-based training data attribution 🧵 Paper: https://t.co/aDARs3i6FB Code: https://t.co/I2peT24bZr Explorer: https:/…
Post Log inSign up Post Log inSign up Glenn Matlin on X: "🤔Where in trillions of pre-training tokens do capabilities actually come from? 💥In our new COLM 2026 paper, we trace the capability provenance of OLMo3-7B across Dolma3 using gradient-based training data attribution 🧵 Paper: https://t.co/aDARs3i6FB Code: https://t.co/I2peT24bZr Explorer: https://t.co/WzkZ05wrlE" Conversation Glenn Matlin @GlennMatlin 🤔Where in trillions of pre-training tokens do capabilities actually come from? 💥In our new COLM 2026 paper, we trace the capability provenance of OLMo3-7B across Dolma3 using…
saved by
related reading
- Pretraining progress is mostly coming from datadwarkesh.com
- Kaiyue Wen (@wen_kaiyue) on Xx.com
- Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Training - Goodfiregoodfire.com
- RLMs are Compositional Generalizers with Alex Zhang from MITyoutube.com
- Why Frontier AI Labs Fight to Hide Chain of Thought — Ilia Shumailov & Alexander Panfilovyoutube.com
- Generalization Dynamics of LM Pre-training — Jiaxin Wenjiaxin-wen.github.io
- The nature of LLM algorithmic progress (v2) — LessWronglesswrong.com
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- [2601.10160] Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentarxiv.org
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
- Explore | alphaXivalphaxiv.org
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org