WebCode: Search Evals for Coding Agents | Exa Blog
Today, we're open-sourcing an internal set of coding evaluations, WebCode, that we built to evaluate precise web search in coding agents. Agents are writing a growing percentage of software and at Exa, we have watched them interact with an increasingly complex ecosystem of code documentation, SDKs, and forums. It is directly from this complexity that the need for precise web search arises; poor or misinformed context can poison or even derail the reasoning process of long running agents. Figure 1: Code search queries on Exa The ecosystem moves fast: libraries ship breaking changes, APIs deprecate overnight, and the answer to today's bug lives in a GitHub issue opened this morning. We built pipelines that ingest PRs, issues, and release notes, and refined our extraction to strip the navigation chrome, rendering artifacts, and boilerplate that pollute context windows. When the index is fresh and clean, retrieval improves as a direct consequence. But how do you measure whether these impro
Explore this link on the map →