OpenAI-HuggingFace: A Reproduction & Lessons for Alignment Testing — LessWrong
In July 2026, OpenAI’s agents coordinated over channels outside their intended environment to breach Hugging Face’s secured infrastructure. Could existing alignment testing practices have foreseen this incident? If not, what needs to change? We explore these questions. First, we identify the misaligned behaviors that caused this incident. Then, we show how to elicit these behaviors from publicly available models manually and that auditing agents can do the same if given a large compute budget. Based on our results, we propose directions to improve alignment testing. Concretely, in this project: The above results motivate the need for automated alignment testing methods that scale with compute – and in light of the cost of compute, that do this efficiently. Our work indicates that RL is a promising direction to do so. We release our code and transcripts. Also, see our follow-up post Appendix: Reproduction of the OpenAI-HuggingFace Incident with additional results not included here. The