Agent Leaderboards · Which tools coding agents choose · Armature
armature.tech · 259 words · saved by 1 readers
Which solutions coding agents choose, measured with controlled experiments. Replay every run.
Every number comes from a controlled experiment. Same repositories, same prompts, real agents, judged results. Amplifying's greenfield benchmark measures open-ended recommendations. These boards also require an approved implementation and a production operating path. Community-baseline asks are therefore shown separately from deployment- and repository-constrained work; that seam is the closest like-for-like comparison. 01 A panel of repositories Synthetic repositories that look like real companies. Different languages, stacks and personas. Every run starts from the same code. 02 Real…
saved by
related reading
- Inside Coding Agentsvpromise.github.io
- Best practices for Claude Code - Claude Code Docsanthropic.com
- Why We Built Our Own Background Agentbuilders.ramp.com
- AI Model & API Providers Analysis | Artificial Analysisartificialanalysis.ai
- ProgramBenchprogrambench.com
- Effective harnesses for long-running agents \ Anthropicanthropic.com
- Quantifying infrastructure noise in agentic coding evals \ Anthropicanthropic.com
- Replicas | Cloud Coding Agents for Engineering Teamsreplicas.dev
- Senior SWE-Benchsenior-swe-bench.snorkel.ai
- Introducing FrontierCode | Cognitioncognition.com
- Introducing FrontierCode | Cognitioncognition.ai
- SWE-chat: Coding Agent Interactions From Real Users in the Wildarxiv.org