MirrorCode: Evidence AI can already do some weeks-long coding tasks | Epoch AI

Early results from MirrorCode benchmark with METR: AI agents can complete weeks-long coding tasks, including reimplementing a 16,000-line codebase.

MirrorCode: Evidence AI can already do some weeks-long coding tasks | Epoch AI Introduction We present early results from MirrorCode , a benchmark (co-developed with METR ) of long-horizon coding tasks derived from real software applications. We find that AI models can autonomously reimplement complex existing software without access to the original program’s source code, provided there is a detailed, checkable specification. For example, Claude Opus 4.6 successfully reimplemented gotree — a bioinformatics toolkit with ~16,000 lines of Go and 40+ commands. We guess this same task would take a

saved by

• Al-Ekram Elahee Hridoy

related reading

• Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver — LessWronglesswrong.com
• BigO(Bench) -- Can LLMs Generate Code with Controlled Time and Space Complexity?alphaxiv.org
• [2501.01257] CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingsarxiv.org
• [2605.02421] AOCI: Symbolic-Semantic Indexing for Practical Repository-Scale Code Understanding with LLMsarxiv.org
• AI excels at code competitions, struggles with real workblog.peterwildeford.com
• Coding Models Are Doing Too Much | whnrehiew.github.io
• When AI builds itself \ Anthropicanthropic.com
• Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebasedatabricks.com
• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METRmetr.org
• Introducing FrontierCode | Cognitioncognition.ai
• Introducing FrontierCode | Cognitioncognition.com
• Measuring AI Ability to Complete Long Tasks - METRmetr.org