Standards for Belief Representations in LLMs

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. As large language models (LLMs) continue to demonstrate remarkable abilities across various domains, computer scientists are developing methods to understand their cognitive processes, particularly concerning how (and if) LLMs internally represent their beliefs about the world. However, this field currently lacks a unified theoretical foundation to underpin the study of belief in LLMs. This article begins filling this gap by proposing adequacy conditions for a representation in an LLM to count as belief-like. We argue that, while the project of belief measurement in LLMs shares striking features with belief measurement as carried out in decision theory and formal epistemology, it also differs in ways that should change how we measure belief. Thus, drawin

Standards for Belief Representations in LLMs Daniel A. Herrmann, Benjamin A. Levinstein Abstract. As large language models (LLMs) continue to demonstrate remarkable abilities across various domains, computer scientists are developing methods to understand their cognitive processes, particularly concerning how (and if) LLMs internally represent their beliefs about the world. However, this field currently lacks a unified theoretical foundation to underpin the study of belief in LLMs. This article begins filling this gap by proposing adequacy conditions for a representation in an LLM to count as

related reading

• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• Believe It or Not: How Deeply do LLMs Believe Implanted Facts?alignment.anthropic.com
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• [2510.17941] Believe It or Not: How Deeply do LLMs Believe Implanted Facts?arxiv.org
• Large Language Models Must Be Taught to Know What They Don't Knowarxiv.org
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
• Cycles of Thought: Measuring LLM Confidence through Stable Explanationsarxiv.org
• [2503.03750] The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemsarxiv.org
• Measuring what Matters: Construct Validity in Large Language Model Benchmarksalphaxiv.org
• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Non-Determinism of “Deterministic” LLM Settingsarxiv.org