Animals vs Ghosts | karpathy

Today's frontier LLM research is not about building animals. It is about summoning ghosts. And a bit more on Sutton's Dwarkesh pod.

Animals vs Ghosts – karpathy Animals vs Ghosts 01 Oct, 2025 Finally had a chance to listen through this Dwarkesh pod with Sutton , which was interesting and amusing. As background, Sutton's " The Bitter Lesson " has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter les

saved by

• Elizabeth Qiu
• Kamal Nayan Bandaru
• Rikard Saqe
• Jirat C
• Julian H
• Tony Kam
• Mapk Uraine

related reading

• Andrej Karpathy on X: "Finally had a chance to listen through this pod with Sutton, which was interesting and amusing. As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinx.com
• Andrej Karpathy — AGI is still a decade awaydwarkesh.com
• 2025 LLM Year in Review – karpathykarpathy.bearblog.dev
• LLM Daydreaming · Gwern.netgwern.net
• The Persona Selection Model: Why AI Assistants might Behave like Humansalignment.anthropic.com
• Richard Sutton – Father of RL thinks LLMs are a dead enddwarkesh.com
• Generalization Dynamics of LM Pre-training — Jiaxin Wenjiaxin-wen.github.io
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
• GenAI Handbookgenai-handbook.github.io
• Dwarkesh Patel on X: "The most interesting part for me is where @karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits thx.com
• Against LLM Reductionism — LessWronglesswrong.com