flâneur

Hidden Technical Debt of AI Systems: Agent Runtime

leehanchung.github.io · 3,321 words · saved by 1 readers

Everyone talks about the model, the prompts, the evaluations. Almost nobody talks about the runtime the agent actually runs in. The runtime is the agent. Scu...

Eleven years ago, Sculley et al. drew the diagram everyone in MLOps has seen: a tiny black box labeled “ML Code” surrounded by a sprawl of much larger boxes — data collection, feature extraction, configuration, monitoring, serving infrastructure. The point of the diagram was that the model code is the smallest piece of a real ML system, and that everything else is where the technical debt accumulates. The same diagram is being redrawn for agents. The agentic model call is the small box. The largest box to the right that is currently driving most of the spend and influencing how system architec

saved by

related reading