Which Future?

michaelnotebook.com · 7,926 words · saved by 3 readers

This essay is the text for a talk on how to wisely navigate risks from transformative technology, especially artificial superintelligence (ASI). It was given at Astera on January 28, 2026. In 1954 the United States carried out its first full-scale test of a thermonuclear bomb, on Bikini Atoll in the Western Pacific. Known as Castle Bravo, the bomb's designers expected a 6 megaton blast. They were shocked when it yielded 15 megatons, an excess of 9 megatons, about 600 times the Hiroshima blast. The unexpected radiation fallout caused many deaths – the exact number is disputed – and serious radiation exposure to more than a thousand people, triggering a major international incident. What went wrong? The bomb contained both the lithium-6 and lithium-7 isotopes of lithium. The bomb's designers believed only the lithium-6 would contribute to the yield, while the lithium-7 would be inert. But the lithium-7 converted to tritium far faster than expected, and that turned out to almost triple th

Which Future? xrisk ai Astera Institute February 17, 2026 How can humanity and posthumanity manage the risks from transformative technologies, such as nuclear weapons, engineered pandemics, artificial superintelligence (ASI), and onward to technologies not yet conceived? This essay argues that sufficiently deep understanding of reality is intrinsically dual-use, so harms inevitably accompany benefits; that market-supplied safety systematically fails for the hardest risks; and that the alignment problem is being badly misconstrued, likely with catastrophic consequences. In particular, I argue t

Which Future?

saved by

related reading