“Next-token predictor” is the wrong mental model for LLMs | Garrin’s Blog
gmcgoldr.github.io · 750 words · saved by 1 readers
“Next-token predictor” is the wrong mental model for LLMs
Edited September 5, 2026. Details below. Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete. It’s a fine zeroth-order approximation, and it is grounded in something real: at inference time, autoregressive language models emit tokens one at a time: while not done: tokens.append(model.sample_next_token(tokens)) This inference loop certainly has the shape of something you might call a next-token predictor. And if we look deeper at what happens during training, prediction is indeed what the model learns to do, at least during pre-training. The…
saved by
related reading
- How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
- How LLMs Actually Work | 0xkato0xkato.xyz
- Simulators — LessWronglesswrong.com
- 2025 LLM Year in Review – karpathykarpathy.bearblog.dev
- Towards a Typology of Strange LLM Chains-of-Thought1a3orn.com
- You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
- Large Language Model: world models or surface statistics?thegradient.pub
- Against LLM Reductionism — LessWronglesswrong.com
- State of RL for reasoning LLMs | A. Weersaweers.de
- Next-Token Predictor Is An AI's Job, Not Its Speciesastralcodexten.com
- A History of Large Language Modelsgregorygundersen.com
- How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com