[2605.31574] Can Generative AI help people navigate Radical Moral Disagreements? The CONSIDER prototype

Abstract:Radical Moral Disagreements (RMDs) are highly polarising topics that are increasingly censored in everyday life, with growing evidence suggesting that this polarisation carries measurable costs to public mental health. To address these challenges, some researchers have proposed Large Language Models (LLMs) as a means to support more democratic deliberation and better moral reasoning. Yet existing tools are poorly calibrated to help people navigate RMDs, because of their intense and divisive characteristics. This paper introduces CONSIDER, a prototype for a one-to-one AI tool for RMD navigation. Drawing on Mill's account of the epistemic value of disagreement, CONSIDER aims at value clarification through structured disagreement with an opposing LLM-generated opinion. We describe CONSIDER's design logic and analyse potential risks posed by such tools to guide future development.

Can Generative AI help people navigate Radical Moral Disagreements? The CONSIDER prototype William Hohnen-Ford1* , Sarah Chen1† , Kathryn B. Francis1,2† , Madeline G. Reinecke1,2† , Ilina Singh1 , David Lyreskog1 . Affiliations 1 University of Oxford, Department of…

saved by

• Vincent Huang

related reading

• [2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USarxiv.org
• Moral Machinemoralmachine.net
• Thoughts on AI in academiatheinfinitesimal.substack.com
• Habermas Machine — LessWronglesswrong.com
• LMCA_dataset.pdfandrew.cmu.edu
• Moral Disagreementplato.stanford.edu
• The Machines Lack Honour — LessWronglesswrong.com
• Sporadicamichaelnotebook.com
• The Best of LessWrong — LessWronglesswrong.com
• Publicationseilab.gatech.edu
• The limits of AI safety via debate — LessWronglesswrong.com
• Dialogue: Is there a Natural Abstraction of Good? — LessWronglesswrong.com