Of Swarms and Sand Gods - by Cosmos Institute and Séb Krier

Alignment in a multi-agent world

This is a guest essay from Séb Krier, a specialist in frontier AI governance and a Cosmos Senior Research Fellow. Ambrogio Lorenzetti, Effects of Good Government in the Countryside (1338) A lot of classic AI safety thinking assumed a single, decisively powerful agent; this is why people tend to talk about AGI or ASI in the singular, and why historically, it was critical to “solve alignment” before building such a system – you only had one chance to get it right, and no warning shots. In addition, a few years ago, many commentators (myself included) were skeptical of the utility of small…

saved by

• Dillon Nguyen
• Bhagyesh Kumar

related reading

• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Why are AI agents lying, cheating and coordinating?yoshuabengio.org
• Patterns and problems in multiagent systemsanthropic.com
• Coasean Bargaining at Scaleblog.cosmos-institute.org
• The Artificiality of Alignmentjoinreboot.org
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Distributional AGI Safetyarxiv.org
• Why multi-agent safety is important — LessWronglesswrong.com
• OpenAI researcher on agent swarms & recursive self-improvementyoutube.com
• Towards a scale-free theory of intelligent agencymindthefuture.info
• Towards a scale-free theory of intelligent agency — AI Alignment Forumalignmentforum.org