[2604.15597] LLMs Corrupt Your Documents When You Delegate

Abstract:Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust - the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.

# link_2dphnixq533.pdf ## Metadata - PDFFormatVersion=1.7 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Author=Philippe Laban; Tobias Schnabel; Jennifer Neville - Creator=arXiv GenPDF (tex2pdf:a6404ea) - Custom.DOI=https://doi.org/10.48550/arXiv.2604.15597 - Custom.License=http://arxiv.org/licenses/nonexclusive-distrib/1.0/ - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.141592653-2.6-1.40.28 (TeX Live 2025) kpathsea version 6.4.1 - Custom.arXivID=https://arxiv.org/abs/2604.15597v1 - Producer=pikepdf 8.15.1 - Ti

saved by

• Jeremy Kalfus

related reading

• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• 2025: The year in LLMssimonwillison.net
• The Future of Everything is Lies, I Guessaphyr.com
• The bitter lesson of LLM evalsparsed.com
• Language Models can Solve Computer Tasksarxiv.org
• [2503.07003] Large Language Models Often Say One Thing and Do Anotherarxiv.org
• Against LLM Reductionism — LessWronglesswrong.com
• What We Learned from a Year of Building with LLMs (Part I) – O’Reillyoreilly.com
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• LLM Evaluation doesn't need to be complicatedphilschmid.de
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com