AI excels at code competitions, struggles with real work

What CodeForces rankings reveal about AI capabilities

AI excels at code competitions, struggles with real work What CodeForces rankings reveal about AI capabilities Peter Wildeford Feb 17, 2025 12 3 Share Photo by Chris Ried on Unsplash Author’s note: This analysis was originally part of a previous post, but it was buried at the bottom and I wanted to feature it more prominently. So I removed it from there, expanded it, and put it here. But sorry if you’re seeing this twice. ~ When IBM's Deep Blue beat chess champion Garry Kasparov in 1997, it was a notable milestone in the development of AI. Today, AI systems are beginning to achieve similar fea

saved by

• Al-Ekram Elahee Hridoy

related reading

• [2501.01257] CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingsarxiv.org
• Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver — LessWronglesswrong.com
• MirrorCode: Evidence AI can already do some weeks-long coding tasks | Epoch AIepoch.ai
• Coding Models Are Doing Too Much | whnrehiew.github.io
• When AI builds itself \ Anthropicanthropic.com
• [2203.07814] Competition-Level Code Generation with AlphaCodearxiv.org
• Competitive programming with AlphaCode — Google DeepMinddeepmind.com
• How AI assistance impacts the formation of coding skills \ Anthropicanthropic.com
• How to fully automate software engineering | Mechanize, Inc.mechanize.work
• claude code is not making your product betterethanding.substack.com
• AI progress is about to speed up | Epoch AIepoch.ai
• When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io