Can AI Learn From Experience? EBR-Bench Results | Epoch AI | Epoch AI
Frontier models show no improvement across 30 playthroughs of the board game Earthborne Rangers, scoring far below expert humans. Epoch AI's EBR-bench probes whether AI can learn on the fly.
Can AI Learn From Experience? EBR-Bench Results | Epoch AI | Epoch AI Can AI systems improve at challenging tasks on the fly, performing them over and over and learning from mistakes? It’s one of the biggest open questions in AI capabilities right now, with large economic and safety implications. Our latest benchmark, EBR-bench, tests AI systems for this ability by having them play Earthborne Rangers, a complex board game, repeatedly. So far, we see little evidence of AI learning from experience. With EBR-bench as part of our benchmarking suite, we have a new tool for detecting if and when tha
saved by
related reading
- The Era of Experience Paper.pdfstorage.googleapis.com
- EdgeBench | Scaling Laws of Environment Learningedge-bench.org
- As Rocks May Think | Eric Jangevjang.com
- How Go Players Disempower Themselves to AI — LessWronglesswrong.com
- Generally capable agents emerge from open-ended playdeepmind.google
- Humans Still Beat AI in the Long Horizon: Revisiting Test-Time Scaling in the Agent Era | Qiuyang Mangjoyemang33.github.io
- RSI Simulator | Paradigmparadigm.xyz
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- TheEraOfExperience.pdfincompleteideas.net
- Voyager | An Open-Ended Embodied Agent with Large Language Modelsvoyager.minedojo.org
- PostTrainBenchposttrainbench.com
- Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver — LessWronglesswrong.com