Scaling in the service of reasoning & model-based ML - Yoshua Bengio
Co-written with my PhD student Edward J. Hu. Scaling seems to work really well. We must be cautious in pursuing research directions that build knowledge…
Co-written with my PhD student Edward J. Hu . Scaling seems to work really well. We must be cautious in pursuing research directions that build knowledge directly into AI systems at the expense of scalability . In fact, even nature builds intelligence on top of large-scale biological neural networks . However, current large-scale systems still exhibit significant factual errors and unpredictable behavior when deployed. While these errors might improve with short-term solutions like more filters, better retrievers, and smarter prompts, these systems do not think like humans do, as indicated by
Explore this link on the map →saved by
related reading
- Building a Reasoning Machine - Ph.D. thesis | Edward Huedwardjhu.com
- Just Ask for Generalization | Eric Jangevjang.com
- The Scaling Hypothesis · Gwern.netgwern.net
- Explore | alphaXivalphaxiv.org
- LLMs and World Models, Part 1 - by Melanie Mitchellaiguide.substack.com
- DeepSeek-R1arxiv.org
- the-illusion-of-thinking.pdfml-site.cdn-apple.com
- The State of LLM Reasoning Model Inferencesebastianraschka.com
- pdfopenreview.net
- World Models: Computing the Uncomputablenotboring.co
- As Rocks May Think | Eric Jangevjang.com
- Generative AI's Act o1: The Reasoning Era Begins | Sequoia Capitalsequoiacap.com