Best explanations of how LLMs work | Roman Vorushin
I maintain a set of best explanations1 of how LLMs train and work. “LLMs” here is a broad term for the frontier models that create value in 2026. They have more components than just a language model, but powerful language models are their necessary core. Universal explainers Considering: humans are universal explainers (Deutsch) - they use creativity to explain the world. Conjecture: LLMs are universal explainers on a non-biological substrate. That’s the opposite of the claim that LLMs are stochastic parrots - that they only imitate, without understanding or creating new knowledge. Creativity Creativity in humans Evolved to understand complex…
I maintain a set of best explanations 1 of how LLMs train and work. “LLMs” here is a broad term for the frontier models that create value in 2026. They have more components than just a language model, but powerful language models are their necessary core. Universal explainers Considering: humans are universal explainers (Deutsch) - they use creativity to explain the world. Conjecture: LLMs are universal explainers on a non-biological substrate. That’s the opposite of the claim that LLMs are stochastic parrots - that they only imitate, without understanding or creating new knowledge. Creativity
saved by
related reading
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- GenAI Handbookgenai-handbook.github.io
- Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
- Animals vs Ghosts – karpathykarpathy.bearblog.dev
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
- LLM Daydreaming · Gwern.netgwern.net
- As Rocks May Think | Eric Jangevjang.com
- How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com
- [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
- Against LLM Reductionism — LessWronglesswrong.com
- Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com