flâneur

AI researchers talk continual learning over a game of HORSE - YouTube

youtube.com · 6,256 words · saved by 1 readers

Follow along using the transcript.

Continual learning, catastrophic forgetting, and distillation vs. RL, discussed over a game of HORSE. Charlie O'Neill, co-head of model training at Baseten, plays Jack Morris, co-founder and head of research at Engram. They cover when to put knowledge in the weights vs. the KV cache, why forgetting might be fine for specialized models, the state of AI research publishing, what evals miss about frontier models, and whether on-policy self-distillation can replace RL. Also, who's the LeBron of AI. Chapters 0:00 Intro and rules 1:17 Learning in weights vs. the KV cache 3:45 Knowledge vs. the…

saved by

related reading