How our new Control Red Team is stress-testing frontier monitors | AISI Work

Early learnings from red-teaming the internal monitors of frontier AI companies, and our perspectives on the open problems that remain.

As LLM agents become more autonomous, they have more opportunities to cause harm. To mitigate risk, frontier AI developers, such as OpenAI and Anthropic , are already deploying agents under the watch of a monitor: a separate LLM that reviews the agent’s actions and flags them if they are dangerous. The growing field of AI control research aims to design these kinds of safety measures, and evaluate whether they would be robust even if agents were trying to evade detection. For over two years, AISI’s Red Team has been evaluating misuse safeguards : protections against humans deliberately eliciti

saved by

• Kaustubh Kislay

related reading

• We need 3rd party Training-Run Assessments — LessWronglesswrong.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Incident Report: unsanctioned agent behaviour during cyber testing | AISI Workaisi.gov.uk
• AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
• Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• AI Control: Improving Safety Despite Intentional Subversion — AI Alignment Forumalignmentforum.org
• Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com
• The Case Against AI Control Research — LessWronglesswrong.com
• Research Areas in AI Control (The Alignment Project by UK AISI) — AI Alignment Forumalignmentforum.org
• The case for ensuring that powerful AIs are controlledblog.redwoodresearch.org
• Frontier Risk Report (February to March 2026) - METRmetr.org