Patterns and problems in multiagent systems \ Anthropic
anthropic.com · 4,238 words · saved by 3 readers
We ran experiments on swarms of Claude agents and found coordination failures, collusion, and sabotage. Here, we share what they mean for AI safety.
Models are improving and AI agents are taking on more tasks in shared codebases, markets, and other social systems. As a result, an increase in real-world interactions between agents is imminent. We've already begun studying this, but still have a lot of uncertainty regarding what this looks like at scale. The trajectory is easy to imagine and hard to slow: current institutions are designed by and for people, resting on assumptions about the sufficiency of oversight at human speed. Some institutions will become human-AI hybrids; others where agents outcompete on speed or cost will become…
saved by
related reading
- Of Swarms and Sand Godsblog.cosmos-institute.org
- Agents of Chaosarxiv.org
- Why multi-agent safety is important — LessWronglesswrong.com
- How we built our multi-agent research system \ Anthropicanthropic.com
- Towards self-driving codebases · Cursorcursor.com
- PhD_thesis_Shirley_Wu_final.pdfcs.stanford.edu
- Agent swarms and the new model economics · Cursorcursor.com
- Getting Up to Speed on Multi-Agent Systems, Part 1: The Landscapechristophermeiklejohn.com
- [2411.00114] Project Sid: Many-agent simulations toward AI civilizationarxiv.org
- The Rise and Fall of Agent Civilizationsdwarkesh.com
- Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale - Microsoft Researchmicrosoft.com
- [2502.14143] Multi-Agent Risks from Advanced AIarxiv.org