CausaLab — Can LLM Agents Discover Causal Mechanisms by Experiment?
A scalable environment that scores not just whether an agent gets the answer, but whether it recovered the causal mechanism — by experimenting like a scientist.
CausaLab — Can LLM Agents Discover Causal Mechanisms by Experiment? TL;DR CausaLab evaluates two things at once : did the agent solve the task, and is its answer grounded in a faithful recovered causal mechanism ? Each episode hides a freshly sampled structural causal model (SCM), so you can't win by reciting memorized causal facts. Prediction ≠ understanding. On observational 6-node graphs, GPT-5.2-high reaches 92% task accuracy but only 0.47 all-edge F 1 . The right number, the wrong graph. How you experiment is the whole game. Observation narrows the hypothesis space; agent-chosen intervent
Explore this link on the map →saved by
related reading
- Faithful, Interpretable Model Explanations via Causal Abstraction | SAIL Blogai.stanford.edu
- [2605.27567] Why LLMs Fail at Causal Discovery and How Interventional Agents Escapearxiv.org
- The machines are fine. I'm worried about us.ergosphere.blog
- LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
- Automated Weak-to-Strong Researcheralignment.anthropic.com
- Causal Scrubbing: a method for rigorously testing interpretability hypotheses [Redwood Research] — LessWronglesswrong.com
- Building Effective AI Agents \ Anthropicanthropic.com
- Agent Evaluation: A Detailed Guidecameronrwolfe.substack.com
- Frederick Eberhardtits.caltech.edu
- How To Become A Mechanistic Interpretability Researcher — AI Alignment Forumalignmentforum.org
- Machine Studying | Jacob Xiaochen Lijacobxli.com
- [2602.06337] Can Post-Training Transform LLMs into Causal Reasoners?arxiv.org