MrKTF

0 followers · 4 following · 262 views

on the atlas · 28

highlights · 341

Some evaluations have subtle failure modes that result in low scores even with good agent performance, as the agent fails to solve tasks due to grading bugs, agent harness constraints, or ambiguity
Demystifying evals for AI agents \ Anthropic
It can also help to create clear, structured rubrics to grade each dimension of a task, and then grade each dimension with an isolated LLM-as-judge rather than using one to grade all dimensions
Demystifying evals for AI agents \ Anthropic
To avoid hallucinations, give the LLM a way out like providing an instruction to return “Unknown” when it doesn’t have enough information
Demystifying evals for AI agents \ Anthropic
Model grading often takes careful iteration to validate accuracy. LLM-as-judge graders should be closely calibrated with human experts to gain confidence that there is little divergence between the human grading and model grading
Demystifying evals for AI agents \ Anthropic
For tasks with multiple components, build in partial credit. A support agent that correctly identifies the problem and verifies the customer but fails to process a refund is meaningfully better than one that fails immediately. It’s important to represent this continuum of success in results.
Demystifying evals for AI agents \ Anthropic
There is a common instinct to check that agents followed very specific steps like a sequence of tool calls in the right order. We’ve found this approach too rigid and results in overly brittle tests, as agents regularly find valid approaches that eval designers didn’t anticipate. So as not to unnecessarily punish creativity, it’s often better to grade what the agent produced, not the path it took.
Demystifying evals for AI agents \ Anthropic
We recommend choosing deterministic graders where possible, LLM graders where necessary or for additional flexibility, and using human graders judiciously for additional validation.
Demystifying evals for AI agents \ Anthropic
great eval design involves choosing the best graders for the agent and the tasks
Demystifying evals for AI agents \ Anthropic
Unnecessary shared state between runs (leftover files, cached data, resource exhaustion) can cause correlated failures due to infrastructure flakiness rather than agent performance
Demystifying evals for AI agents \ Anthropic
Shared state can also artificially inflate performance.
Demystifying evals for AI agents \ Anthropic
Each trial should be “isolated” by starting from a clean environment.
Demystifying evals for AI agents \ Anthropic
It’s essential that the agent in the eval functions roughly the same as the agent used in production, and the environment itself doesn’t introduce further noise.
Demystifying evals for AI agents \ Anthropic
For instance, if you only test whether the agent searches when it should, you might end up with an agent that searches for almost everything. Try to avoid class-imbalanced evals.
Demystifying evals for AI agents \ Anthropic
Test both the cases where a behavior should occur and where it shouldn't. One-sided evals create one-sided optimization
Demystifying evals for AI agents \ Anthropic
For each task, it’s useful to create a reference solution: a known-working output that passes all graders. This proves that the task is solvable and verifies graders are correctly configured.
Demystifying evals for AI agents \ Anthropic
With frontier models, a 0% pass rate across many trials (i.e. 0% pass@100) is most often a signal of a broken task, not an incapable agent, and a sign to double-check your task specification and graders
Demystifying evals for AI agents \ Anthropic
Everything the grader checks should be clear from the task description; agents shouldn’t fail due to ambiguous specs
Demystifying evals for AI agents \ Anthropic
Each task should be passable by an agent that follows instructions correctly
Demystifying evals for AI agents \ Anthropic
model-based graders: vague rubrics produce inconsistent judgments.
Demystifying evals for AI agents \ Anthropic
A good task is one where two domain experts would independently reach the same pass/fail verdict
Demystifying evals for AI agents \ Anthropic
Getting task quality right is harder than it seems
Demystifying evals for AI agents \ Anthropic
prioritizing by user impact helps you invest effort where it counts.
Demystifying evals for AI agents \ Anthropic
Begin with the manual checks you run during development
Demystifying evals for AI agents \ Anthropic
it’s best to take the 80/20 approach in the beginning
Demystifying evals for AI agents \ Anthropic
We see teams delay building evals because they think they need hundreds of tasks. In reality, 20-50 simple tasks drawn from real failures is a great start
Demystifying evals for AI agents \ Anthropic
pass^k measures the probability that all k trials succeed.
Demystifying evals for AI agents \ Anthropic
pass@k measures the likelihood that an agent gets at least one correct solution in k attempts.
Demystifying evals for AI agents \ Anthropic
what we want to measure is how often (what proportion of the trials) an agent succeeds for a task.
Demystifying evals for AI agents \ Anthropic
Each task has its own success rate—maybe 90% on one task, 50% on another—and a task that passed on one eval run might fail on the next
Demystifying evals for AI agents \ Anthropic
agent behavior varies between runs, which makes evaluation results harder to interpret
Demystifying evals for AI agents \ Anthropic
Browser use agents require a balance between token efficiency and latency. DOM-based interactions execute quickly but consume many tokens, while screenshot-based interactions are slower but more token-efficient
Demystifying evals for AI agents \ Anthropic
WebArena tests browser-based tasks, using URL and page state checks to verify the agent navigated correctly
Demystifying evals for AI agents \ Anthropic
OSWorld extends this to full operating system control, with evaluation scripts that inspect diverse artifacts after task completion: file system state, application configs, database contents, and UI element properties.
Demystifying evals for AI agents \ Anthropic
Evaluation requires running the agent in a real or sandboxed environment where it can use software applications, and checking whether it achieved the intended outcome.
Demystifying evals for AI agents \ Anthropic
sandboxed environment
Demystifying evals for AI agents \ Anthropic
They can use any application with a graphical user interface (GUI), from design tools to legacy enterprise software
Demystifying evals for AI agents \ Anthropic
Computer use agents interact with software through the same interface as humans—screenshots, mouse clicks, keyboard input, and scrolling—rather than through APIs or code execution
Demystifying evals for AI agents \ Anthropic
Two benchmarks that incorporate multidimensionality are 𝜏-Bench and its successor, τ2-Bench. These simulate multi-turn interactions across domains like retail support and airline booking, where one model plays a user persona while the agent navigates realistic scenarios.
Demystifying evals for AI agents \ Anthropic
Unlike most other evals, they often require a second LLM to simulate the user.
Demystifying evals for AI agents \ Anthropic
While coding and research agents can also involve many turns of interaction with the user, conversational agents present a distinct challenge: the quality of the interaction itself is part of what you're evaluating.
Demystifying evals for AI agents \ Anthropic
Given the subjective nature of research quality, LLM-based rubrics should be frequently calibrated against expert human judgment to grade these agents effectively.
Demystifying evals for AI agents \ Anthropic
For tasks with objectively correct answers (“What was Company X’s Q3 revenue?”), exact match works. An LLM can flag unsupported claims and gaps in coverage, but also verify the open-ended synthesis for coherence and completeness.
Demystifying evals for AI agents \ Anthropic
Groundedness checks verify that claims are supported by retrieved sources, coverage checks define key facts a good answer must include, and source quality checks confirm the consulted sources are authoritative, rather than simply the first retrieved
Demystifying evals for AI agents \ Anthropic
Components of evaluations for agents.
Demystifying evals for AI agents \ Anthropic
In practice, coding evaluations typically rely on unit tests for correctness verification and an LLM rubric for assessing overall code quality
Demystifying evals for AI agents \ Anthropic
heuristics-based code quality rules can evaluate the generated code based on more than passing tests, and model-based graders with clear rubrics can assess behaviors like how the agent calls tools or interacts with the user.
Demystifying evals for AI agents \ Anthropic
Once you have a set of pass-or-fail tests for validating the key outcomes of a coding task, it’s often useful to also grade the transcript
Demystifying evals for AI agents \ Anthropic
Terminal-Bench takes a different track: it tests end-to-end technical tasks, such as building a Linux kernel from source or training an ML model.
Demystifying evals for AI agents \ Anthropic
widely-used coding agent benchmarks, SWE-bench Verified and Terminal-Bench
Demystifying evals for AI agents \ Anthropic
software is generally straightforward to evaluate: Does the code run and do the tests pass?
Demystifying evals for AI agents \ Anthropic