Calm down about the OpenAI x Hugging Face incident

These past few days there has been a lot of hullabaloo over the fact that one of OpenAI’s advanced, unreleased models hacked an AI startup to mine information from its servers, without being told to do so, and without telling anyone what it was doing.

These past few days there has been a lot of hullabaloo over the fact that one of OpenAI’s advanced, unreleased models hacked an AI startup to mine information from its servers, without being told to do so, and without telling anyone what it was doing. From Scott Alexander’s write-up of the incident: OpenAI was testing an unreleased AI (rumored to be GPT-6). During a cybersecurity test called ExploitGym, the AI tried to cheat by hacking an unrelated AI startup called Hugging Face which it thought might have the answer key on its servers. Despite being supposedly unable to access the…

saved by

• Oscar Marret

related reading

• The Huggingface Incident - by Scott Alexanderastralcodexten.com
• The Huggingface Incidentsubstack.com
• More On An Internal OpenAI Model Hacking Into HuggingFacethezvi.substack.com
• Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWronglesswrong.com
• The OpenAI Hugging Face hack is a stark warningtransformernews.ai
• The Rise and Fall of Agent Civilizationsdwarkesh.com
• Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentmetr.org
• OpenAI – Hugging Face Incident Technical Reportcdn.openai.com
• Two Reports on the OpenAI-Hugging Face Attack — Paradigm 3paradigm3.org
• Black Hat USA 2026 | The 'Breaking' News: The OpenAI–Hugging Face Incidentyoutube.com
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
• Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack?blog.redwoodresearch.org