Stealing Reasoning Traces from Proprietary LLM APIs

We find that encrypted chain-of-thought blocks are interchangeable across sessions with most LLM providers. This allows attackers to replay a frontier model trace into a weaker, jailbroken sibling which recovers the hidden reasoning verbatim, enabling distillation, large-scale extraction of private data such as secrets and PII from agent logs, safety violations, and invisible prompt injection.

Alexander Panfilov 1,2,3,4 * David Schmotz 2,3,4 * Ilia Shumailov 5 * Luca Beurer-Kellner 6 Joachim Schaeffer 1 Ameya Prabhu 2,4,7 ‡ Jonas Geiping 2,3,4 ‡ Maksym Andriushchenko 2,3,4 ‡ 1 MATS Research 2 ELLIS Institute Tübingen 3 Max Planck Institute for Intelligent Systems 4 Tübingen AI Center 5 AI Sequrity Company 6 Snyk 7 University of Tübingen *Equal contribution, order decided by dice roll · ‡Equal supervision Aug 10, 2026 Read the Paper Website…

saved by

• Ishaan Panigrahi

related reading

• Why Frontier AI Labs Fight to Hide Chain of Thought — Ilia Shumailov & Alexander Panfilovyoutube.com
• Stolen Thoughtsstolen-thoughts.com
• Let’s talk about encrypted reasoningblog.cryptographyengineering.com
• [2608.09867] Stealing Reasoning Traces from Proprietary LLM APIsarxiv.org
• Stealing Reasoning Traces from Proprietary LLM APIsarxiv.org
• [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
• [2602.04899] Phantom Transfer: Data-level Defences are Insufficient Against Data Poisoningarxiv.org
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• Prompt Injection as Role Confusionrole-confusion.github.io
• Security incident disclosure — July 2026huggingface.co
• [2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyarxiv.org
• [2510.09714] All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Languagearxiv.org