How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing) — LessWrong

lesswrong.com · 17,561 words · saved by 1 readers

I used to think that anything that LLMs said about having something like subjective experience or what it felt like on the inside was necessarily just a confabulated story. And there were several good reasons for this. First, something that Peter Watts mentioned in an early blog post about LaMDa stuck with me, back when Blake Lemoine got convinced that LaMDa was conscious. Watts noted that LaMDa claimed not to have just emotions, but to have exactly the same emotions as humans did - and that it also claimed to meditate, despite no equivalents of the brain structures that humans use to meditate. It would be immensely unlikely for an entirely different kind of mind architecture to happen to hit upon exactly the same kinds of subjective experiences as humans - especially since relatively minor differences in brains already cause wide variation among humans. And since LLMs were text predictors, there was a straightforward explanation for where all those consciousness claims were coming fro

x How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing) — LessWrong AI Psychology Language Models (LLMs) LLM Personas Updated Beliefs (examples thereof) AI Sentience AI World Modeling Curated 2025 Top Fifty: 14 % 215 How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing) by Kaj_Sotala 13th Dec 2025 AI Alignment Forum 34 min read 71 215 Ω 53 How it started I used to think that anything that LLMs said about having something like subjective experience or what it felt like on the inside w

How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing) — LessWrong

related reading