The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind - YouTube
youtube.com · 10,094 words · saved by 1 readers
Follow along using the transcript. Follow along using the transcript. New New New New New New New New Dubbed New New New
Slides: https://drive.google.com/file/d/1xMohjQcTmQuUd_OiZ3hB1r47WB1WM3Am/view https://x.com/jxmnop/status/1877761437931581798 - the screenshot thumbnail hey everyone welcome back to a len space sliming pod this is alesio partner in C deible and I'm joined by my host swix founder of small hey and today we are very happy to have fishy Aral from gdm what is your what is your role at gdm um a staff research scientist research yeah staff research scientist and occasionally you do these talks uh it sounds like you do them internally but then also you publish your slides uh and caught my eye on…
saved by
related reading
- Noam Shazeer and Jack Rae: Scaling Test-time Compute, Reactions to Ilya & AGIyoutube.com
- On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
- Deep Dive into LLMs like ChatGPTyoutube.com
- Lil'Loglilianweng.github.io
- The Art of Scaling Reinforcement Learning for LLMs | Sean Bell, ResolveAI Labsyoutube.com
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- Distillation Walkthroughvladfeinberg.com
- SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io
- RLMs are Compositional Generalizers with Alex Zhang from MITyoutube.com
- Efficient LLM inferencefinbarrtimbers.substack.com
- What I learned this week - Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RLdwarkesh.com