✳flâneur — a map of the web's best reading

Preparing for the Intelligence Explosion | Forethought

forethought.org · 21,716 words · saved by 6 readers

AI that can accelerate research could drive a century of technological progress over just a few years. During such a period, new technological or political developments will raise consequential and hard-to-reverse decisions, in rapid succession. We call these developments *grand challenges*. These challenges include new weapons of mass destruction, AI-enabled autocracies, races to grab offworld resources, and digital beings worthy of moral consideration, as well as opportunities to dramatically improve quality of life and collective decision-making. We argue that these challenges cannot always be delegated to future AI systems, and suggest things we can do today to meaningfully improve our prospects. AGI preparedness is therefore not just about ensuring that advanced AI systems are aligned: we should be preparing, now, for the disorienting range of developments an intelligence explosion would bring.

Preparing for the Intelligence Explosion William MacAskill Fin Moorhouse Authors Citations Cite Citations PDF Contact 11th March 2025 Preparing for the Intelligence Explosion Abstract 1. Introduction 2. A century in a decade The last century, compressed Asymmetric acceleration The accelerated decade 3. The intelligence explosion and beyond Progress in AI capabilities AI-human cognitive parity The technology explosion The industrial explosion 4. Grand challenges AI takeover Highly destructive technologies Power-concentrating mechanisms Value lock-in mechanisms AI agents and digital minds Space

Explore this link on the map →

saved by

related reading