Why I Hope AGI Kills Us - by Noah Birnbaum
Note: This article is somewhat of a troll - I don’t accept the premises, but I do think that both the conclusions follow and that this should count as a reductio ad absurdum for at least one of the premises given. On the other hand, one can definitely just accept the premises and bite the bullet on the conclusion (though I would probably call you crazy!). Also, I will largely be assuming that qualia evolving as part of evolution and that hedonic utilitarianism or preference utilitarianism are true because there is too much to be said about those here. One can certainly reasonably disagree with these premises, but I’m just showing what I take the conclusions to be given that you do accept these premises (or accept the premises to some degree). The Orthogonality Thesis, created and popularized by Nick Bostrom, is the thesis that asserts that there can exist arbitrarily intelligent agents pursuing any kind of goal. This post is, in some sense (more to be said about why this is more compli
Note: This article is somewhat of a troll - I don’t accept the premises, but I do think that both the conclusions follow and that this should count as a reductio ad absurdum for at least one of the premises given. On the other hand, one can definitely just accept the premises and bite the bullet on the conclusion (though I would probably call you crazy!). Also, I will largely be assuming that qualia evolving as part of evolution and that hedonic utilitarianism or preference utilitarianism are true because there is too much to be said about those here. One can certainly reasonably disagree…
related reading
- Orthogonality Thesis — LessWronglesswrong.com
- AGI Ruin: A List of Lethalities — LessWronglesswrong.com
- Beliefs are Chosen to Serve Goals — LessWronglesswrong.com
- Nick Bostrom’s Home Pagenickbostrom.com
- Varieties Of Doomminihf.com
- After Orthogonality: Virtue-Ethical Agency and AI Alignmentthegradient.pub
- Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
- Heading Toward Morality — LessWronglesswrong.com
- Suicidal Compassion: How Utilitarianism at AI Companies Endangers Humanityai-frontiers.org
- Unfalsifiable stories of doom | Mechanize, Inc.mechanize.work
- Prolegomena to a Theory of Fun — LessWronglesswrong.com
- The Best of LessWrong — LessWronglesswrong.com