Scaling in the service of reasoning & model-based ML - Yoshua Bengio

Co-written with my PhD student Edward J. Hu. Scaling seems to work really well. We must be cautious in pursuing research directions that build knowledge…

Co-written with my PhD student Edward J. Hu . Scaling seems to work really well. We must be cautious in pursuing research directions that build knowledge directly into AI systems at the expense of scalability . In fact, even nature builds intelligence on top of large-scale biological neural networks . However, current large-scale systems still exhibit significant factual errors and unpredictable behavior when deployed. While these errors might improve with short-term solutions like more filters, better retrievers, and smarter prompts, these systems do not think like humans do, as indicated by

saved by

• Stephen Lu

related reading

• Building a Reasoning Machine - Ph.D. thesis | Edward Huedwardjhu.com
• The Scaling Hypothesis · Gwern.netgwern.net
• Scaling is subtler than it seemsberen.io
• Will scaling work?dwarkeshpatel.com
• Just Ask for Generalization | Eric Jangevjang.com
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• Q-learning is not yet scalableseohong.me
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• Spending Inference Time - Kevin Lukevinlu.ai
• Scaling works. These researchers are betting billions it isn't enoughsubstack.com
• machine learning imindslice.substack.com