We May be Surprised Again: Why I take LLMs seriously.
"Deep Learning is Easy, Learn something Harder" - I proclaimed in one of my early and provocative blog posts from 2016. While some observations were fair, that post is now evidence that I clearly underestimated the the impact simple techniques will have, and probably gave counterproductive advice. I wasn't alone
March 22, 2023 We May be Surprised Again: Why I take LLMs seriously. "Deep Learning is Easy, Learn something Harder" - I proclaimed in one of my early and provocative blog posts from 2016. While some observations were fair, that post is now evidence that I clearly underestimated the impact simple techniques will have, and probably gave counterproductive advice. I wasn't alone in my deep learning skepticism, in fact I'm far from being the most extreme deep learning skeptic. Many of us who grew up working in Bayesian ML, convex optimization, kernels and statistical learning theory confidently pr
saved by
related reading
- As Rocks May Think | Eric Jangevjang.com
- How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
- DeepSeek-R1arxiv.org
- GenAI Handbookgenai-handbook.github.io
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- The Unintelligibility is Ours: Notes on Chain of Thought1a3orn.com
- Against LLM Reductionism — LessWronglesswrong.com
- The necessity of machine learning theory in mitigating AI riskmishabelkin.substack.com
- The Future of Everything is Lies, I Guessaphyr.com
- Thoughts on AI in academiatheinfinitesimal.substack.com
- Things we learned about LLMs in 2024simonwillison.net