Have We Seen an Acceleration in Discoveries? - METR
Tom Cunningham and Nate Rush look at public time series of discoveries in cyber, math, and algorithms to see whether LLMs have accelerated the discovery rate.
Q: How much has discovery accelerated? LLMs have shown the ability to make novel discoveries across many domains. How much has this affected the aggregate discovery rate? In the figures below we plot all the data sources we can find, and make some very loose observations: Discovery of cyber vulnerabilities has accelerated sharply. Discovery of math results has accelerated somewhat. However this is harder to objectively measure. Discovery of optimizations has not shown a dramatic acceleration. These conclusions are based only on public discoveries. It is quite plausible that AI labs are…
saved by
related reading
- Mathematicians may be worried, but AI-for-science is going to be great, recursively self-improving, and we’re going to learn loadssimondedeo.com
- Evidence on AI R&D Progress from NanoGPT - METRmetr.org
- Shtetl-Optimized >> Blog Archive >> Dispatches from the possibly last days of human relevancescottaaronson.blog
- As Rocks May Think | Eric Jangevjang.com
- 2025: The year in LLMssimonwillison.net
- Mathematics in the Library of Babel - Daniel Littdaniellitt.com
- There Are No New Ideas in AI… Only New Datasetsblog.jxmo.io
- I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
- rsrch spacersrch.space
- My picture of the present in AI — LessWronglesswrong.com
- rsrch spacersrch.space
- The least understood driver of AI progress | Epoch AIepoch.ai