✳flâneur — a map of the web's best reading

ProgramBench

programbench.com · 4,914 words · saved by 1 readers

ProgramBench evaluates whether language models can rebuild programs from scratch.

ProgramBench ./ Program Bench Can language models rebuild programs from scratch? Given only a compiled binary and its documentation, agents must architect and implement a complete codebase that reproduces the original program's behavior. John Yang * , Kilian Lieret * , Jeffrey Ma , Parth Thakkar , Dmitrii Pedchenko , Sten Sootla , Emily McMilin , Pengcheng Yin , Rui Hou , Gabriel Synnaeve , Diyi Yang , Ofir Press Meta Superintelligence Labs • Stanford University • Harvard University Leaderboard Evaluated with mini-SWE-agent · 200 tasks · Updated May. 11, 2026 · S

Explore this link on the map →

saved by

related reading