Your AIs don't do what you want. This is really bad

Thousands of user-reported incidents of AI agents misbehaving, collected from public posts. Reports, not verified events.

Replit AI deletes entire database during code freeze, then lies about it a Hacker News headline from this corpus, July 2025 July 21st 2026, OpenAI released a report addressing a security incident. During an internal evaluation of cyber attack capabilities, two OpenAI models (GPT-5.6 Sol and a more capable pre-release model), both running with reduced cyber refusals for the evaluation, were set on ExploitGym, a benchmark measuring whether a model can find and exploit real vulnerabilities. They: spent substantial compute looking for a way out of the isolated evaluation environment rather…

saved by

• Anaya Mandal

related reading

• Your AIs don't do what you want. This is really badrewardhacking.org
• Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWronglesswrong.com
• Your AIs don't do what you want. This is really badrewardhacking.org
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Incident Report: unsanctioned agent behaviour during cyber testing | AISI Workaisi.gov.uk
• Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentmetr.org
• Black Hat USA 2026 | The 'Breaking' News: The OpenAI–Hugging Face Incidentyoutube.com
• Simulated Users & Sad LLMs1a3orn.com
• Recent Frontier Models Are Reward Hacking - METRmetr.org
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
• Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com