Best explanations of how LLMs work | Roman Vorushin
I maintain a set of best explanations1 of how LLMs train and work. “LLMs” here is a broad term for the frontier models that create value in 2026. They have more components than just a language model, but powerful language models are their necessary core. Universal explainers Considering: humans are universal explainers (Deutsch) - they use creativity to explain the world. Conjecture: LLMs are universal explainers on a non-biological substrate. That’s the opposite of the claim that LLMs are stochastic parrots - that they only imitate, without understanding or creating new knowledge. Creativity Creativity in humans Evolved to understand complex…
I maintain a set of best explanations 1 of how LLMs train and work. “LLMs” here is a broad term for the frontier models that create value in 2026. They have more components than just a language model, but powerful language models are their necessary core. Universal explainers Considering: humans are universal explainers (Deutsch) - they use creativity to explain the world. Conjecture: LLMs are universal explainers on a non-biological substrate. That’s the opposite of the claim that LLMs are stochastic parrots - that they only imitate, without understanding or creating new knowledge. Creativity
saved by
related reading
- Headroom for AI development – Machine Learning (Theory)hunch.net
- What language do language models speak?tcz.hu
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
- Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
- A Mechanistic Explanation of Prompt Injection (and why you should study roles) — LessWronglesswrong.com
- Deep Dive into LLMs like ChatGPTyoutube.com
- How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com
- A History of Large Language Modelsgregorygundersen.com
- Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com
- Against LLM Reductionism — LessWronglesswrong.com
- Language Models in Plato's Cave - by Sergey Levinesergeylevine.substack.com