GBA Eval - Build a Game Boy Advance emulator in WebAssembly from scratch
gbaeval.com · 151 words · saved by 1 readers
Frontier AI coding agents try to write a Game Boy Advance emulator from scratch. Their emulators are graded against Mesen2.
GBA EvalFromMechanize, Inc. Models try to write a Game Boy Advance emulator from scratch. We give frontier AI coding agents 24 hours to write a complete GBA emulator with WebAssembly support. We grade the emulators against Mesen2, one of the most accurate software GBA emulators available. GBA Eval is similar to the work Mechanize does with top AI labs to make environments for training and evaluating frontier LLMs. We hope it gives a concrete sense of what we mean by “environments” and “grading”. If designing and building high-quality evaluations sounds like something you'd want to do,…
saved by
related reading
- Mechanize, Inc.mechanize.work
- Demystifying evals for AI agents \ Anthropicanthropic.com
- FrontierSWEfrontierswe.com
- FrontierSWEfrontierswe.com
- Senior SWE-Benchsenior-swe-bench.snorkel.ai
- ProgramBenchprogrambench.com
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- Introducing FrontierCode | Cognitioncognition.ai
- Introducing FrontierCode | Cognitioncognition.com
- Harness design for long-running application developmentanthropic.com
- TERMINAL-BENCHtbench.ai
- Introducing SWE-grep and SWE-grep-mini: RL for Multi-Turn, Fast Context Retrieval | Cognitioncognition.ai