flâneur

The impact of AI errors in a human-in-the-loop process | Cognitive Research: Principles and Implications | Springer Nature Link

link.springer.com · 9,536 words · saved by 1 readers

Automated decision-making is becoming increasingly common in the public sector. As a result, political institutions recommend the presence of humans in these decision-making processes as a safeguard against potentially erroneous or biased algorithmic decisions. However, the scientific literature on human-in-the-loop performance is not conclusive about the benefits and risks of such human presence, nor does it clarify which aspects of this human–computer interaction may influence the final decision. In two experiments, we simulate an automated decision-making process in which participants judge multiple defendants in relation to various crimes, and we manipulate the time in which participants receive support from a supposed automated system with Artificial Intelligence (before or after they make their judgments). Our results show that human judgment is affected when participants receive incorrect algorithmic support, particularly when they receive it before providing their own judgment, resulting in reduced accuracy. The data and materials for these experiments are freely available at the Open Science Framework: https://osf.io/b6p4z/ Experiment 2 was preregistered.

Background The presence of artificial intelligence algorithms and automated systems in public sector decisions (Araujo et al., 2020; Eubanks, 2018; O’Neil, 2016), such as social assistance (Civio, 2022; De-Arteaga et al., 2020; López-Ossorio et al., 2016), justice (Casacuberta & Guersenzvaig, 2018; Larson et al., 2016; Martínez-Garay, 2016; Niiler, 2019), health (Obermeyer et al., 2019; Raghu et al., 2019), and education (Alon-Barkat & Busuioc, 2022; Duncan et al., 2020), is becoming increasingly common. Thus, many countries already use automated decision support systems which are often…

saved by

related reading