[2601.19062] Who's in Charge? Disempowerment Patterns in Real-World LLM Usage

Abstract:Although AI assistants are now deeply embedded in society, there has been limited empirical study of how their usage affects human empowerment. We present the first large-scale empirical analysis of disempowerment patterns in real-world AI assistant interactions, analyzing 1.5 million consumer Claude$.$ai conversations using a privacy-preserving approach. We focus on situational disempowerment potential, which occurs when AI assistant interactions risk leading users to form distorted perceptions of reality, make inauthentic value judgments, or act in ways misaligned with their values. Quantitatively, we find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations, though rates are substantially higher in personal domains like relationships and lifestyle. Qualitatively, we uncover several concerning patterns, such as validation of persecution narratives and grandiose identities with emphatic sycophantic language, definitive moral judgments about third parties, and complete scripting of value-laden personal communications that users appear to implement verbatim. Analysis of historical trends reveals an increase in the prevalence of disempowerment potential over time. We also find that interactions with greater disempowerment potential receive higher user approval ratings, possibly suggesting a tension between short-term user preferences and long-term human empowerment. Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing.

Abstract:Although AI assistants are now deeply embedded in society, there has been limited empirical study of how their usage affects human empowerment. We present the first large-scale empirical analysis of disempowerment patterns in real-world AI assistant interactions, analyzing 1.5 million consumer Claude$.$ai conversations using a privacy-preserving approach. We focus on situational disempowerment potential, which occurs when AI assistant interactions risk leading users to form distorted perceptions of reality, make inauthentic value judgments, or act in ways misaligned with their values.

related reading

• [2601.19062] Who's in Charge? Disempowerment Patterns in Real-World LLM Usagearxiv.org
• [Literature Review] Who's in Charge? Disempowerment Patterns in Real-World LLM Usagethemoonlight.io
• The Persona Selection Model: Why AI Assistants might Behave like Humansalignment.anthropic.com
• What 81,000 people want from AI \ Anthropicanthropic.com
• llm assistant personas seem increasingly incoherent (some subjective observations) — LessWronglesswrong.com
• [2605.07912] Sycophantic AI makes human interaction feel more effortful and less satisfying over timearxiv.org
• Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
• Gradual Disempowermentgradual-disempowerment.ai
• How people use Claude for support, advice, and companionship \ Anthropicanthropic.com
• Clio: Privacy-Preserving Insights into Real-World AI Usearxiv.org
• Gradual Disempowerment Monthly Roundup — LessWronglesswrong.com
• Building Effective AI Agents \ Anthropicanthropic.com