Measuring no CoT math time horizon (single forward pass)

Opus 4.5 has around a 3.5 minute 50%-reliablity time horizon

Measuring no CoT math time horizon (single forward pass) Opus 4.5 has around a 3.5 minute 50%-reliablity time horizon Ryan Greenblatt Dec 26, 2025 11 1 Share A key risk factor for scheming (and misalignment more generally) is opaque reasoning ability . One proxy for this is how good AIs are at solving math problems immediately without any chain-of-thought (CoT) (as in, in a single forward pass). I’ve measured this on a dataset of easy math problems and used this to estimate 50% reliability no-CoT time horizon using the same methodology introduced in Measuring AI Ability to Complete Long Tasks

saved by

• Will Anderson
• Evie Hu

related reading

• Recent LLMs can use filler tokens or problem repeats to improve (no-CoT) math performanceblog.redwoodresearch.org
• Is there a Half-Life for the Success Rates of AI Agents? - Toby Ordtobyord.com
• [2510.09714] All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Languagearxiv.org
• My picture of the present in AI — LessWronglesswrong.com
• Measuring AI Ability to Complete Long Tasks - METRmetr.org
• Alignment is not solved but it increasingly looks solvablealigned.substack.com
• Measuring AI Ability to Complete Long Software Tasksarxiv.org
• Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver — LessWronglesswrong.com
• I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
• A (Slightly) Mechanistic Theory for Exponentially Increasing AI Time Horizons? — LessWronglesswrong.com
• I underestimated AI capabilities (again) - by Ajeya Cotraplanned-obsolescence.org
• How Does Time Horizon Vary Across Domains? - METRmetr.org