flâneur

Contra shard theory, in the context of the diamond maximizer problem — LessWrong

lesswrong.com · 6,562 words · saved by 1 readers

A bunch of my response to shard theory is a generalization of how niceness is unnatural. In a similar fashion, the other “shards” that the shard theory folk want to learn are unnatural too. That said, I'll spend a few extra words responding to the admirably-concrete diamond maximizer proposal that TurnTrout recently published, on the theory that briefly gesturing at my beliefs is better than saying nothing. I’ll be focusing on the diamond maximizer plan, though this criticism can be generalized and applied more broadly to shard theory. Finally, I'll note that the diamond maximization problem is not in fact the problem "build an AI that makes a little diamond", nor even "build an AI that probably makes a decent amount of diamond, while also spending lots of other resources on lots of other stuff" (although the latter is more progress than the former). The diamond maximization problem (as originally posed by MIRI folk) is a challenge of building an AI that definitely optimizes for a part

x Contra shard theory, in the context of the diamond maximizer problem — LessWrong 2022 MIRI Alignment Discussion Shard Theory AI Frontpage 107 Contra shard theory, in the context of the diamond maximizer problem by So8res 13th Oct 2022 AI Alignment Forum 3 min read 19 107 Ω 48 A bunch of my response to shard theory is a generalization of how niceness is unnatural . In a similar fashion, the other “shards” that the shard theory folk want to learn are unnatural too. That said, I'll spend a few extra words responding to the admirably-concrete diamond maximizer proposal that TurnTrout recently pu

related reading