benchmarks.bio — Agentic AI benchmarks on messy, real-world biological data
Can frontier AI agents reason about real, messy biological data? SpatialBench (159 evals, 5 spatial transcriptomics platforms) is live, with deterministic graders that verify the key biological result. scBench for single-cell RNA-seq coming soon. By LatchBio.
Agentic benchmarks on messy, real-world tasks Each problem includes a snapshot of real experimental data taken immediately prior to a target decision or analysis step, a description of the task through a high-level scientific lens, and a deterministic grader (e.g., Jaccard similarity of sets) that evaluates recovery of the key biological result in a verifiable manner. The benchmark is designed to test durable biological reasoning rather than method-specific implementation details and require empirical interaction with the data. Metric Balanced score — Trial-weighted harmonic mean of…
saved by
related reading
- BenchmarkList: Track the Frontier of AI Capabilitiesbenchmarklist.com
- TERMINAL-BENCHtbench.ai
- Quantifying infrastructure noise in agentic coding evals \ Anthropicanthropic.com
- Claude SWE-Bench Performance \ Anthropicanthropic.com
- PostTrainBenchposttrainbench.com
- AI Model & API Providers Analysis | Artificial Analysisartificialanalysis.ai
- GitHub - openai/mle-bench: MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineeringgithub.com
- Paving the way for agents in biology \ Anthropicanthropic.com
- ProgramBenchprogrambench.com
- ReactBenchreact-bench.com
- Senior SWE-Benchsenior-swe-bench.snorkel.ai
- EdgeBench | Scaling Laws of Environment Learningedge-bench.org