The impact of AI errors in a human-in-the-loop process | Cognitive Research: Principles and Implications | Springer Nature Link
Automated decision-making is becoming increasingly common in the public sector. As a result, political institutions recommend the presence of humans in these decision-making processes as a safeguard against potentially erroneous or biased algorithmic decisions. However, the scientific literature on human-in-the-loop performance is not conclusive about the benefits and risks of such human presence, nor does it clarify which aspects of this human–computer interaction may influence the final decision. In two experiments, we simulate an automated decision-making process in which participants judge multiple defendants in relation to various crimes, and we manipulate the time in which participants receive support from a supposed automated system with Artificial Intelligence (before or after they make their judgments). Our results show that human judgment is affected when participants receive incorrect algorithmic support, particularly when they receive it before providing their own judgment, resulting in reduced accuracy. The data and materials for these experiments are freely available at the Open Science Framework: https://osf.io/b6p4z/ Experiment 2 was preregistered.
Background The presence of artificial intelligence algorithms and automated systems in public sector decisions (Araujo et al., 2020; Eubanks, 2018; O’Neil, 2016), such as social assistance (Civio, 2022; De-Arteaga et al., 2020; López-Ossorio et al., 2016), justice (Casacuberta & Guersenzvaig, 2018; Larson et al., 2016; Martínez-Garay, 2016; Niiler, 2019), health (Obermeyer et al., 2019; Raghu et al., 2019), and education (Alon-Barkat & Busuioc, 2022; Duncan et al., 2020), is becoming increasingly common. Thus, many countries already use automated decision support systems which are often…
saved by
related reading
- AI and prediction of criminal justice risk: revisiting the COMPAS dataset, experiments, and the problem of algorithmic bias and fairnesslink.springer.com
- When is automated decision making legitimate?fairmlbook.org
- The Myth of the Human-in-the-Loop and the Reality of Cognitive Offloading - Perry World Houseperryworldhouse.upenn.edu
- Beyond explainability: justifiability and contestability of Algorithmic Decision Systemsinria.hal.science
- Constitutional AI: Harmlessness from AI Feedbackarxiv.org
- Oversight Assistants: Turning Compute into Understandingbounded-regret.ghost.io
- [2605.07912] Sycophantic AI makes human interaction feel more effortful and less satisfying over timearxiv.org
- Three Challenges for AI-Assisted Decision-Making - PMCpmc.ncbi.nlm.nih.gov
- [1911.03020] A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairnessarxiv.org
- [2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USarxiv.org
- 2306.09983arxiv.org
- AI Debaters are More Persuasive when Arguing in Alignment with Their Own Beliefsarxiv.org