[2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety

Abstract:AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability.

Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety Tomek Korbak∗ UK AI Security Institute Mikita Balesni∗ Apollo Research Elizabeth Barnes METR Yoshua Bengio University of Montreal & Mila Joe Benton Anthropic Joseph Bloom UK AI Security Institute arXiv:2507.11473v2 [cs.AI] 7…

saved by

• Saul Munn
• Lydia Nottingham
• Alex Yun
• Bhagyesh Kumar

related reading

• [2512.18311] Monitoring Monitorabilityarxiv.org
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationarxiv.org
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• The fragile foundations of CoT monitoring | Christopher Pottsweb.stanford.edu
• [2512.11949] Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorsarxiv.org
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• How AI Is Learning to Think in Secretnickandresen.substack.com
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• When Chain of Thought is Necessary, Language Models Struggle to Evade Monitorsarxiv.org
• [2603.05706] Reasoning Models Struggle to Control their Chains of Thoughtarxiv.org
• [2510.09714] All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Languagearxiv.org