“Next-token predictor” is the wrong mental model for LLMs | Garrin’s Blog
gmcgoldr.github.io · 750 words · saved by 1 readers
“Next-token predictor” is the wrong mental model for LLMs
Edited September 5, 2026. Details below. Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete. It’s a fine zeroth-order approximation, and it is grounded in something real: at inference time, autoregressive language models emit tokens one at a time: while not done: tokens.append(model.sample_next_token(tokens)) This inference loop certainly has the shape of something you might call a next-token predictor. And if we look deeper at what happens during training, prediction is indeed what the model learns to do, at least during pre-training. The…
saved by
related reading
- LLM Resourcesforrestbicker.com
- How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
- How LLMs Actually Work | 0xkato0xkato.xyz
- GenAI Handbookgenai-handbook.github.io
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com
- State of RL for reasoning LLMs | A. Weersaweers.de
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- The Llama Hitchiking Guide to Local LLMs – hackerllamaosanseviero.github.io
- Large Language Model: world models or surface statistics?thegradient.pub
- LLMs can invent their own compression - Rajan Agarwalrajan.sh
- Large Language Models explained briefly | 3Blue1Brown3blue1brown.com