Algorithmic Content Selection and the Impact of User Disengagement

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. Digital services face a fundamental trade-off in content selection: they must balance the immediate revenue gained from high-reward content against the long-term benefits of maintaining user engagement. Traditional multi-armed bandit models assume that users remain perpetually engaged, failing to capture the possibility that users may disengage when dissatisfied, thereby reducing future revenue potential. In this work, we introduce a model for the content selection problem that explicitly accounts for variable user engagement and disengagement. In our framework, content that maximizes immediate reward is not necessarily optimal in terms of fostering sustained user engagement. Our contributions are twofold. First, we develop computational and statistical

Algorithmic Content Selection and the Impact of User Disengagement Algorithmic Content Selection and the Impact of User Disengagement Emilio Calvano University of Rome Nika Haghtalab Stanford University Ellen Vitercik University of California, Berkeley Eric Zhao Stanford University Abstract Digital services face a fundamental trade-off in content selection: they must balance the immediate revenue gained from high-reward content against the long-term benefits of maintaining user engagement. Traditional multi-armed bandit models assume that users remain perpetually engaged, failing to capture th

related reading

• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• Reconciling the accuracy-diversity trade-off in recommendationsarxiv.org
• Effect of recommending users and opinions on the network connectivity and idea generation processarxiv.org
• user_interactions.pdfself-distillation.github.io
• Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?arxiv.org
• The Multi-Armed Bandit Problem and Its Solutions | Lil'Loglilianweng.github.io
• An Analysis of Intent-Based Marketsarxiv.org
• [arxiv] an algorithmic framework for fairness elicitationarxiv.org
• The Bayes Banditfrancesco215.github.io
• [2503.10990] Statistical Impossibility and Possibility of Aligning LLMs with Human Preferences: From Condorcet Paradox to Nash Equilibriumarxiv.org
• Exploration for the Efficient Deployment of Reinforcement Learning Agentsopenreview.net
• [2506.22401] Exploration from a Primal-Dual Lens: Value-Incentivized Actor-Critic Methods for Sample-Efficient Online RLarxiv.org