Measuring no CoT math time horizon (single forward pass)

Opus 4.5 has around a 3.5 minute 50%-reliablity time horizon

Measuring no CoT math time horizon (single forward pass) Opus 4.5 has around a 3.5 minute 50%-reliablity time horizon Ryan Greenblatt Dec 26, 2025 11 1 Share A key risk factor for scheming (and misalignment more generally) is opaque reasoning ability . One proxy for this is how good AIs are at solving math problems immediately without any chain-of-thought (CoT) (as in, in a single forward pass). I’ve measured this on a dataset of easy math problems and used this to estimate 50% reliability no-CoT time horizon using the same methodology introduced in Measuring AI Ability to Complete Long Tasks

saved by

• Will Anderson
• Evie Hu

related reading

• Recent LLMs can use filler tokens or problem repeats to improve (no-CoT) math performanceblog.redwoodresearch.org
• My picture of the present in AI — LessWronglesswrong.com
• As Rocks May Think | Eric Jangevjang.com
• Mathematics in the Library of Babel - Daniel Littdaniellitt.com
• I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• AI progress is about to speed up | Epoch AIepoch.ai
• Measuring AI Ability to Complete Long Tasks - METRmetr.org
• How Does Time Horizon Vary Across Domains? - METRmetr.org
• Measuring AI Ability to Complete Long Software Tasksarxiv.org
• AI Timelines — LessWronglesswrong.com
• Inside the Secret Meeting Where Mathematicians Struggled to Outsmart AI | Scientific Americanscientificamerican.com