Hidden Technical Debt of AI Systems: Agent Runtime
Everyone talks about the model, the prompts, the evaluations. Almost nobody talks about the runtime the agent actually runs in. The runtime is the agent. Scu...
Eleven years ago, Sculley et al. drew the diagram everyone in MLOps has seen: a tiny black box labeled “ML Code” surrounded by a sprawl of much larger boxes — data collection, feature extraction, configuration, monitoring, serving infrastructure. The point of the diagram was that the model code is the smallest piece of a real ML system, and that everything else is where the technical debt accumulates. The same diagram is being redrawn for agents. The agentic model call is the small box. The largest box to the right that is currently driving most of the spend and influencing how system architec
saved by
related reading
- How to Harness Coding Agents with the Right Infrastructure | Blogalexlavaee.me
- A Taxonomy of RL Environments for LLM Agentsleehanchung.github.io
- A basic systems architecture for AI agents that do autonomous research — LessWronglesswrong.com
- Building Effective AI Agents \ Anthropicanthropic.com
- Arjun Virkarjunvirk.com
- Low Latency and Model Training at Modalrhea24.github.io
- AddyOsmani.com - Long-running Agentsaddyosmani.com
- Scaling Managed Agents: Decoupling the brain from the hands \ Anthropicanthropic.com
- What we’ve learned building cloud agents · Cursorcursor.com
- Why We Built Our Own Background Agentbuilders.ramp.com
- Effective harnesses for long-running agents \ Anthropicanthropic.com
- Harness design for long-running application developmentanthropic.com