Paretotopian Goal Alignment - by Eric Drexler

From tribes to states, competition for limited resources has driven conflict, and while resource competition is only one cause of conflict, it remains a powerful force.1 Within a fixed-resource framework, interests appear to be directly opposed, because dividing fixed resources is a zero-sum game. If we take this zero-sum game as a model of the future, incentives for cooperation seem sharply limited. The concept of “Paretotopian goal alignment” shows how prospects for AI-enabled capabilities could reshape incentives — provided that these prospects and their implications are understood in advance. This article illustrates how prospects for greatly expanded resources could blunt incentives for conflict and help align the goals of competing actors towards mutually beneficial outcomes. The division of fixed resources is often viewed as a zero-sum game: For one party to gain resources, another must lose. Within the scope of a resource-centered framework,2 a zero-sum constraint creates direc

From tribes to states, competition for limited resources has driven conflict, and while resource competition is only one cause of conflict, it remains a powerful force.1 Within a fixed-resource framework, interests appear to be directly opposed, because dividing fixed resources is a zero-sum game. If we take this zero-sum game as a model of the future, incentives for cooperation seem sharply limited. The concept of “Paretotopian goal alignment” shows how prospects for AI-enabled capabilities could reshape incentives — provided that these prospects and their implications are understood in advan