flâneur

Epoch Capabilities Index | Epoch AI

epoch.ai · 334 words · saved by 1 readers

The Epoch Capabilities Index combines many benchmarks into a single capability scale for comparing models over time.

The general ECI is a composite metric which uses scores from over 50 distinct benchmarks to generate a single, general capability scale. At a high level, ECI stitches together its component benchmarks, determining their relative difficulty by making comparisons wherever models are evaluated on multiple benchmarks. Individual models obtain higher ECI scores if they perform better on harder benchmarks. We give an overview of our methodology here; further technical details are available in our paper, A Rosetta Stone for AI Benchmarks, which was funded by Google DeepMind, and written in…

saved by

related reading