SotA ARC-AGI-2 Results with REPL Agents | Symbolica Blog
symbolica.ai · 2,387 words · saved by 2 readers
Exploiting the reasoning capabilities of code mode agents and RLMs with the Agentica framework.
The Agentica framework by Symbolica improves ARC-AGI-2 performance across three frontier models. The approach builds on code-mode agents [ 1 ] and Recursive Language Models (RLMs) [ 2 ]. Our implementation achieves a score of 85.28% with Opus 4.6 (120k) High and increase the scores of GPT 5.2 (XHigh) and Opus 4.5 by 10 and 20 percentage points respectively. The agent is 350 lines of Python and uses the Agentica framework . Check out the code on GitHub symbolica-ai/arcgentica Submission Public Eval Score (%) Cost ($/task) Agentica Opus 4.6 (120k) High 85.28 6.94 Opus 4.6 (120k) High 79.03 3.81
saved by
related reading
- LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
- Building Effective AI Agents \ Anthropicanthropic.com
- Building Effective AI Agents \ Anthropicanthropic.com
- Getting 50% (SoTA) on ARC-AGI with GPT-4o — LessWronglesswrong.com
- Prime Agent: A self-improving RLM agentprimeintellect.ai
- Getting 50% (SoTA) on ARC-AGI with GPT-4oblog.redwoodresearch.org
- Prime Agent: A Self-Improving RLM Harnessarxiv.org
- Agent Evaluation: A Detailed Guidecameronrwolfe.substack.com
- Quantifying infrastructure noise in agentic coding evals \ Anthropicanthropic.com
- OpenAI o3 Breakthrough High Score on ARC-AGI-Pub | ARC Prizearcprize.org
- [2605.02421] AOCI: Symbolic-Semantic Indexing for Practical Repository-Scale Code Understanding with LLMsarxiv.org
- OpenAI o1 Results on ARC-AGI-Pub | ARC Prizearcprize.org