Now What? A Recipe for After the Problem Setting (in the Agentic Age) | Tom Silver
tomsilver.github.io · 1,710 words · saved by 1 readers
The research recipe I follow after defining a problem setting, with a new step for the agentic age: implement the pure agent baseline.
A follow-up to What’s Your Problem? An Oft-Missing Section in AI Papers. A while back, I argued that AI papers should include a Problem Setting section. My litmus test was whether a reader could implement a Problem abstract class, a Method abstract class, and a run function that takes both and returns metrics. That post was nominally about writing, but really it was about thinking. What class of problems do you care about? What is a method allowed to know and do? How will you judge performance? This post picks up where that one left off. You have defined the problem setting. Now what?…
saved by
related reading
- Automated Weak-to-Strong Researcheralignment.anthropic.com
- Demystifying evals for AI agents \ Anthropicanthropic.com
- Building Effective AI Agents \ Anthropicanthropic.com
- A Pragmatic Vision for Interpretability — AI Alignment Forumalignmentforum.org
- Agent Evaluation: A Detailed Guidecameronrwolfe.substack.com
- f316275b44ee2de533102913828a8107-Paper-Datasets_and_Benchmarks_Track.pdfproceedings.neurips.cc
- An Apple-Picking Model of AI R&D | Tom Cunningham – Tom Cunninghamtecunningham.github.io
- AI agents can't yet do open-ended AI researchnormaltech.ai
- How To Think Real Good | Meta-rationalitymetarationality.com
- 2023 letter | Zhengdongzhengdongwang.com
- Effective harnesses for long-running agents \ Anthropicanthropic.com
- Make It Work, Then Prove It Works: A Framework for Research - Vincent Sitzmannvincentsitzmann.com