Cooperative AI
In this area, we would like to see proposals for work on security challenges that arise as advanced and interacting AI systems become more widespread throughout society. Often these challenges will overlap somewhat with our other research areas. Sign up to our mailing list to stay informed about upcoming initiatives © Cooperative AI Foundation 2025 The Cooperative AI Foundation is registered with the Charity Commission for England and Wales under charity number 1201294, and incorporated as a company limited by guarantee established in England for charitable purposes only under company number 13485176. Its registered address is Courtenay House, Pynes Hill, Exeter, EX2 5AZ.
Cooperative AI ← Back to Overview Multi-Agent Security What It Is and Why It Matters In this area, we would like to see proposals for work on security challenges that arise as advanced and interacting AI systems become more widespread throughout society. Often these challenges will overlap somewhat with our other research areas. Specific Work We Would Like to Fund Assessing what security vulnerabilities advanced multi-agent systems have that single-agent systems do not, and developing defence strategies for these vulnerabilities, such as improvements in network design, communication protocol d
related reading
- Patterns and problems in multiagent systemsanthropic.com
- [2502.14143] Multi-Agent Risks from Advanced AIarxiv.org
- Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
- Oxford Witt Labwittlab.ai
- Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
- Distributional AGI Safetyarxiv.org
- Why multi-agent safety is important — LessWronglesswrong.com
- The End-State Fallacy: Where Is AI Security Headed?endstatefallacy.com
- Student Projects / Supervision | Oxford Witt Labwittlab.ai
- Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
- Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale - Microsoft Researchmicrosoft.com
- Of Swarms and Sand Godsblog.cosmos-institute.org