We May be Surprised Again: Why I take LLMs seriously.
"Deep Learning is Easy, Learn something Harder" - I proclaimed in one of my early and provocative blog posts from 2016. While some observations were fair, that post is now evidence that I clearly underestimated the the impact simple techniques will have, and probably gave counterproductive advice. I wasn't alone
March 22, 2023 We May be Surprised Again: Why I take LLMs seriously. "Deep Learning is Easy, Learn something Harder" - I proclaimed in one of my early and provocative blog posts from 2016. While some observations were fair, that post is now evidence that I clearly underestimated the impact simple techniques will have, and probably gave counterproductive advice. I wasn't alone in my deep learning skepticism, in fact I'm far from being the most extreme deep learning skeptic. Many of us who grew up working in Bayesian ML, convex optimization, kernels and statistical learning theory confidently pr
saved by
related reading
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- Just Ask for Generalization | Eric Jangevjang.com
- How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
- Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
- A History of Large Language Modelsgregorygundersen.com
- Scaling is subtler than it seemsberen.io
- [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
- Deep Dive into LLMs like ChatGPTyoutube.com
- Against LLM Reductionism — LessWronglesswrong.com
- Thoughts on AI in academiatheinfinitesimal.substack.com