Against LLM Reductionism
erichgrunewald.com · 7,103 words · saved by 1 readers
Texts on philosophy, poetry, literature, history, altruism, science, programming and music.
Against LLM Reductionism Against LLM Reductionism 2023-03-08 • 25 min read • comment via LW , EAF Summary # Large language models (henceforth, LLMs) are sometimes said to be “just” shallow pattern matchers, “just” massive look-up tables or “just” autocomplete engines. These comparisons amount to a form of (methodological) reductionism. While there’s some truth to them, I think they smuggle in corollaries that are either false or at least not obviously true. For example, they seem to imply that what LLMs are doing amounts merely to rote memorisation and/or clever parlour tricks, and that they c
related reading
- Against LLM Reductionism — LessWronglesswrong.com
- As Rocks May Think | Eric Jangevjang.com
- Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- LLM Daydreaming · Gwern.netgwern.net
- Language Models in Plato's Cave - by Sergey Levinesergeylevine.substack.com
- GenAI Handbookgenai-handbook.github.io
- The Future of Everything is Lies, I Guessaphyr.com
- Large Language Model: world models or surface statistics?thegradient.pub
- How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com
- Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org