✳flâneur — a map of the web's best reading

Dreams of Friendliness — LessWrong

lesswrong.com · 9,151 words · saved by 1 readers

Yesterday I described three classes of deep problem with qualitative-physics-like strategies for building nice AIs - e.g., the AI is reinforced by smiles, and happy people smile, therefore the AI will tend to act to produce happiness. In shallow form, three instances of the three problems would be: And the deep forms of the problem are, roughly: But there are other ways, and deeper ways, of viewing the failure of qualitative-physics-based Friendliness strategies. Every now and then, someone proposes the Oracle AI strategy: "Why not just have a superintelligence that answers human questions, instead of acting autonomously in the world?" Sounds pretty safe, doesn't it? What could possibly go wrong? Well... if you've got any respect for Murphy's Law, the power of superintelligence, and human stupidity, then you can probably think of quite a few things that could go wrong with this scenario. Both in terms of how a naive implementation could fail - e.g., universe tiled with tiny users a

x Dreams of Friendliness — LessWrong AI Boxing (Containment) AI Risk Personal Blog 29 Dreams of Friendliness by Eliezer Yudkowsky 31st Aug 2008 11 min read 81 29 Continuation of : Qualitative Strategies of Friendliness Yesterday I described three classes of deep problem with qualitative-physics-like strategies for building nice AIs - e.g., the AI is reinforced by smiles, and happy people smile, therefore the AI will tend to act to produce happiness . In shallow form, three instances of the three problems would be: Ripping people's faces off and wiring them into smiles; Building lots of tiny ag

Explore this link on the map →

related reading