Coding vs thinking — Paradigm 3
We’re interested in the prospects for (presumably safer) narrow AI staying competitive, instead of general systems; Cursor’s Composer coding finetune of Kimi is probably the most intense attempt to specialise a general model: probably more than 10^25 FLOPs of post-training; We find that, compared to its base model, Composer shows…
TL;DR # We’re interested in the prospects for (presumably safer) narrow AI staying competitive, instead of general systems. Cursor’s Composer coding finetune of Kimi is probably the most intense attempt to specialise a general model: probably more than 10^25 FLOPs of post-training. We find that, compared to its base model, Composer shows significant gains (+20% to 60%) on visual reasoning benchmarks, RPG-style games, and (as you’d hope) agentic coding. But, surprisingly, we also saw severe losses (-30% to -40%) on mathematical and scientific reasoning benchmarks. This cuts against the old idea
saved by
related reading
- Training Composer for longer horizons · Cursorcursor.com
- Composer2.pdfcursor.com
- Improving Composer through real-time RL · Cursorcursor.com
- Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebasedatabricks.com
- How we compare model quality in Cursorcursor.com
- Kimi K3 Tech Blog: Open Frontier Intelligencekimi.com
- [2203.07814] Competition-Level Code Generation with AlphaCodearxiv.org
- Coding Models Are Doing Too Much | whnrehiew.github.io
- AI progress is about to speed up | Epoch AIepoch.ai
- Introducing FrontierCode | Cognitioncognition.ai
- Introducing FrontierCode | Cognitioncognition.com
- [2501.01257] CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingsarxiv.org