RLMs are Compositional Generalizers with Alex Zhang from MIT - YouTube
youtube.com · 11,483 words · saved by 1 readers
Follow along using the transcript.
we're back with Alex L. Zhang, MIT CSAIL 2nd year PhD student and creator of Recursive Language Models, to talk about compositional generalization and locally in-distribution. as a refresher RLMs put the prompt in a python REPL instead of feeding it to the model, letting it peek, grep, chunk and recursively call itself on the pieces. the new update is that an RLM trained on short tasks only can generalizes to tasks 8-32x longer and even to new domains, while the vanilla transformer stays can't. we get into what "locally in-distribution" really means, alex's mismanaged geniuses…
saved by
related reading
- Recursive Language Models — Alex Zhang, MIT PhDyoutube.com
- Language model harnesses are compositional generalizersalexzhang13.github.io
- How AI is Changing Programmingyoutube.com
- AI researchers debate how close we are to recursive self-improvementyoutube.com
- Next-Latent Prediction Transformers Learn World Models feat. Jayden Teoh from MITyoutube.com
- RL Scaling Laws for LLMs - by Cameron R. Wolfe, Ph.D.cameronrwolfe.substack.com
- Alex L. Zhangalexzhang13.github.io
- Recursive Language Models | Alex L. Zhangalexzhang13.github.io
- How GPT, Claude, and Gemini are actually trained and served – Reiner Popeyoutube.com
- How trillion-parameter AI models work and how to serve themyoutube.com
- How LLMs Actually Work | 0xkato0xkato.xyz
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com