Safe Pareto Improvements Research Agenda — Center on Long-Term Risk
Safe Pareto improvements (SPIs) are modifications to agents’ bargaining strategies that make all parties better off, regardless of their original strategies.
Executive summary Safe Pareto improvements (SPIs) are ways of changing agents' bargaining strategies that make all parties better off, regardless of their original strategies. SPIs are an unusually robust approach to preventing catastrophic conflict between AI systems, especially AIs capable of credible commitments. This is because SPIs can reduce the costs of conflict without shifting bargaining power, or requiring agents to agree on what counts as "fair". Despite their appeal, SPIs aren't guaranteed to be adopted. AIs or humans in the loop might lock in SPI-incompatible commitments, or…
saved by
related reading
- A gap in the theoretical justification for surrogate goals and safe Pareto improvements – The Universe from an Intentional Stancecasparoesterheld.com
- Cooperation, Conflict, and Transformative Artificial Intelligence: A Research Agenda — Center on Long-Term Risklongtermrisk.org
- Automated Weak-to-Strong Researcheralignment.anthropic.com
- Commitment ability in multipolar AI scenarios — Center on Long-Term Risklongtermrisk.org
- The Case Against AI Control Research — LessWronglesswrong.com
- How do we (more) safely defer to AIs? — LessWronglesswrong.com
- Efficient tradeoffs and the safety-usefulness tradeoff model — LessWronglesswrong.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- Measurement Research Agenda – Center on Long-Term Risklongtermrisk.org
- Coasean Bargaining at Scaleblog.cosmos-institute.org
- AI Deterrence by Betrayalaibetrayal.com
- Sections 1 & 2: Introduction, Strategy and Governance — LessWronglesswrong.com