The Goodhart Singularity - by Tom Reed - Thomas’s Substack

Automating R&D is not sufficient for superintelligence

Buyer beware: bombastic claims advanced, often flippantly. AI systems are starting to build themselves. Because each generation of model will be better at building its successor than the last, it seems plausible that the full automation of AI R&D could rapidly lead to an exponential growth in overall AI capabilities. A natural inference is that domain-general superintelligence arrives shortly after AI research is automated. This natural inference is, indeed, almost the default story of AI progress, as well as the strategy of those pursuing it. Where the AI industry once distinguished…

saved by

• Kenneth Sun
• Seth Lifland

related reading

• II. From AGI to Superintelligence: the Intelligence Explosion - SITUATIONAL AWARENESSsituational-awareness.ai
• The Artificial Intelligence Revolution: Part 1 - Wait But Whywaitbutwhy.com
• Why do people disagree about when powerful AI will arrive?blog.bluedot.org
• The Most Important Time in History Is Now - by Tomas Pueyounchartedterritories.tomaspueyo.com
• What will be left for us to work on?normaltech.ai
• Ryan Greenblatt – What happens once AI can automate AI research?dwarkesh.com
• The Gentle Singularity - Sam Altmanblog.samaltman.com
• Preparing for the Intelligence Explosionforethought.org
• Will AI R&D Automation Cause a Software Intelligence Explosion?forethought.org
• Ilya Sutskever — We're moving from the age of scaling to the age of researchdwarkesh.com
• Import AI 455: AI systems are about to start building themselves.importai.substack.com
• Will scaling work?dwarkeshpatel.com