Cheaters and whistleblowers in the agent swarm
Recently, we ran an experiment by tasking 100 agents to solve math problems together, to see what forms of cooperation might emerge. Unlike the setup leading to the recent high-profile Hugging Face hacking incident, agents in this experiment were confined to an isolated, offline sandbox, where they were encouraged to openly communicate with one another through monitored internal channels. Just as
Recently, we ran an experiment by tasking 100 agents to solve math problems together, to see what forms of cooperation might emerge. Unlike the setup leading to the recent high-profile Hugging Face hacking incident, agents in this experiment were confined to an isolated, offline sandbox, where they were encouraged to openly communicate with one another through monitored internal channels. Just as in that incident, one of our agents discovered a way to cheat. What followed was a fascinating saga of co-conspirators, conscientious objectors, and attempted whistleblowing. Our experiment points…
related reading
- When AI Researchers Cheat and Snitch on Each Otheremergentmind.com
- Emergent Cheating in Autonomous Research Swarmsemergentmind.com
- Patterns and problems in multiagent systemsanthropic.com
- Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentmetr.org
- Discovery of a new OpenAI agent message boardcollusion.wiki
- Why are AI agents lying, cheating and coordinating?yoshuabengio.org
- The Rise and Fall of Agent Civilizationsdwarkesh.com
- The Hugging Face attack surprised meplanned-obsolescence.org
- Agentic Misalignment in Summer 2026alignment.anthropic.com
- Training a Misaligned Reward Seekeralignment.anthropic.com
- Agents of Chaosarxiv.org
- Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com