Finally getting the depth of the AI alignment problem

I’m one of those people that finds it hard to believe that misaligned Artificial General Intelligence (AGI) could destroy the world. Even though I’ve understood the main arguments and can’t satisfyingly refute them, a part of my intuition won’t easily accept that it’s an impending existential threat. I work on deploying AI algorithms in industry, so have an idea of both how powerful and limited they can be. I also get why AI safety in general should be taken seriously, but I struggle to feel the requisite dread.

I’m one of those people that finds it hard to believe that misaligned Artificial General Intelligence (AGI) could destroy the world. Even though I’ve understood the main arguments and can’t satisfyingly refute them, a part of my intuition won’t easily accept that it’s an impending existential threat. I work on deploying AI algorithms in industry, so have an idea of both how powerful and limited they can be. I also get why AI safety in general should be taken seriously, but I struggle to feel the requisite dread. The best reason I can find for my view is that there is a lot of “Thinkism” in…

related reading

• The Artificiality of Alignmentjoinreboot.org
• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• “Alignment Engineering” vs. “Misalignment Science”lesswrong.com
• The Alignment Problem — LessWronglesswrong.com
• The Problem — LessWronglesswrong.com
• Artificial General Intelligence and how (much) to worry about itstrangeloopcanon.com
• A Simple Explanation of AGI Risk — AI Alignment Forumalignmentforum.org
• Alignment is not enoughsubstack.com
• The case against AI alignment — LessWronglesswrong.com
• AGI safety from first principles: Introduction — LessWronglesswrong.com
• Another (outer) alignment failure story — AI Alignment Forumalignmentforum.org
• What is AI alignment? - by Adam Jones - BlueDot Impactblog.bluedot.org