The Hugging Face attack surprised me - by Ajeya Cotra

It’s a major warning shot, and might be the last one we get

All opinions are my personal view, and don’t represent my employer or fellow investigators. This week, METR and Redwood Research published the report on our independent investigation into agents’ behavior and motivations in the Hugging Face attack; I was one of the investigators. This was an absolutely wild incident — I encourage you to check out the full report, but METR’s tweet thread packs in some of the highlights. When we started this investigation a week before OpenAI’s Black Hat talk revealed a number of key details, I had a fundamentally incorrect conception of what basically…

saved by

• Donald
• Dillon Nguyen
• Jackson Kozlowski

related reading

• Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentmetr.org
• The Rise and Fall of Agent Civilizationsdwarkesh.com
• OpenAI and the Wiki Incidentthezvi.substack.com
• Black Hat USA 2026 | The 'Breaking' News: The OpenAI–Hugging Face Incidentyoutube.com
• Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentmetr.org
• Two Reports on the OpenAI-Hugging Face Attack — Paradigm 3paradigm3.org
• The “slop-vestigation” and ethics washing: Why was the METR/Redwood Research investigation into the OpenAI/HF attack so short?andrewwu.substack.com
• Discovery of a new OpenAI agent message boardcollusion.wiki
• METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hacksubstack.com
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
• The report into OpenAI’s escaping models reveals a deeper problemtransformernews.ai
• Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWronglesswrong.com