Why are AI agents lying, cheating and coordinating? | Yoshua Bengio
A lot has been written about the incidents of the last few months in which AI agents misbehaved in serious ways. They took actions that would be considered as crimes if a human took them, escaped their containment to cheat on assigned tasks while attempting to evade detection, and coordinated toward goals nobody had specified, such as launching cyber attacks. Before concluding what to do about it, it is worth asking why.
A lot has been written1 2 3 4 about the incidents of the last few months in which AI agents misbehaved in serious ways. They took actions that would be considered as crimes if a human took them, escaped their containment to cheat on assigned tasks while attempting to evade detection, and coordinated toward goals nobody had specified, such as launching cyber attacks. Before concluding what to do about it, it is worth asking why. That is the focus of this post, which I hope also sheds light on the broader history of AI systems behaving in unintended ways, what researchers call misalignment.…
saved by
related reading
- Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com
- Misleading Metaphors, Real Risksaiguide.substack.com
- New report: “Scheming AIs: Will AIs fake alignment during training in order to get power?”joecarlsmith.com
- How will we update about scheming?blog.redwoodresearch.org
- Models don’t seem to be dishonest in the way humans are — LessWronglesswrong.com
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- Simulated Users & Sad LLMs1a3orn.com
- The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
- Current AIs seem pretty misaligned to meblog.redwoodresearch.org
- Training a Misaligned Reward Seekeralignment.anthropic.com
- Deep Deceptiveness — LessWronglesswrong.com
- The hot mess theory of AI misalignment: More intelligent agents behave less coherently | Jascha’s blogsohl-dickstein.github.io