flâneur

Mirrors and Paintings — LessWrong

lesswrong.com · 11,253 words · saved by 1 readers

Background: There's a proposal for Friendly AI called "Coherent Extrapolated Volition" which I don't really want to divert the discussion to, right now. Among many other things, CEV involves pointing an AI at humans and saying (in effect) "See that? That's where you find the base content for self-renormalizing morality." Hal Finney commented on the Pebblesorter parable: I wonder what the Pebblesorter AI would do if successfully programmed to implement [CEV]... Would the AI pebblesort? Or would it figure that if the Pebblesorters got smarter, they would see that pebblesorting was pointless and arbitrary? Would the AI therefore adopt our own parochial morality, forbidding murder, theft and sexual intercourse among too-young people? Would that be the CEV of Pebblesorters? I imagine we would all like to think so, but it smacks of parochialism, of objective morality. I can't help thinking that Pebblesorter CEV would have to include some aspect of sorting pebbles. Doesn't that sugges

x Mirrors and Paintings — LessWrong Coherent Extrapolated Volition Reflective Reasoning Metaethics Personal Blog 31 Mirrors and Paintings by Eliezer Yudkowsky 23rd Aug 2008 10 min read 42 31 Followup to : Sorting Pebbles Into Correct Heaps , Invisible Frameworks Background: There's a proposal for Friendly AI called "Coherent Extrapolated Volition" which I don't really want to divert the discussion to, right now. Among many other things, CEV involves pointing an AI at humans and saying (in effect) "See that? That's where you find the base content for self-renormalizing morality." Hal Finney com

related reading