Quantifying Truesight With SAEs · Gwern.net

Proposal to use SAEs to crack open the dark matter of LLM inference about text.

--- title: Quantifying Truesight With SAEs description: "Proposal to use SAEs to crack open the dark matter of LLM inference about text." created: 2025-05-25 modified: 2025-05-26 status: finished importance: 7 confidence: possible css-extension: dropcaps-kanzlei ... > LLMs know more than they say, like who wrote some text. > How can we find out these known unknowns of how much they *really* know, when their internals are a mysterious scrambled blackbox? > > But we can possibly use sparse autoencoders to turn their internal thoughts into a long list of simpler properties summarizing what the LL

saved by

• Yudhister Joel Kumar
• Julian H
• Will Anderson
• Bhagyesh Kumar

related reading

• Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
• [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
• Writing for LLMs So They Listen · Gwern.netgwern.net
• [2607.05316] How Much is Left? LLMs Linearly Encode Their Remaining Output Lengtharxiv.org
• [2510.17941] Believe It or Not: How Deeply do LLMs Believe Implanted Facts?arxiv.org
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• LLM Daydreaming · Gwern.netgwern.net
• A gentle introduction to sparse autoencoders — LessWronglesswrong.com
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelsarxiv.org