Against LLM Reductionism - LessWrong
SUMMARY * Large language models (henceforth, LLMs) are sometimes said to be "just" shallow pattern matchers, "just" massive look-up tables or "just" autocomplete engines. These comparisons amount to…
x Against LLM Reductionism — LessWrong Interpretability (ML & AI) Language Models (LLMs) AI Frontpage 140 Against LLM Reductionism by Erich_Grunewald 8th Mar 2023 Linkpost for www.erichgrunewald.com 21 min read 17 140 Summary Large language models (henceforth, LLMs) are sometimes said to be "just" shallow pattern matchers, "just" massive look-up tables or "just" autocomplete engines. These comparisons amount to a form of (methodological) reductionism. While there's some truth to them, I think they smuggle in corollaries that are either false or at least not obviously true. For example, they se
saved by
related reading
- Against LLM Reductionismerichgrunewald.com
- Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
- Large Language Model: world models or surface statistics?thegradient.pub
- Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
- You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
- Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com
- Deep Dive into LLMs like ChatGPTyoutube.com
- Productizing Large Language Modelsblog.replit.com
- A History of Large Language Modelsgregorygundersen.com
- The nature of LLM algorithmic progress (v2) — LessWronglesswrong.com
- Best explanations of how LLMs work | Roman Vorushinvorushin.github.io
- How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com