When is automated decision making legitimate?

A student is proud of the creative essay she wrote for a standardized test. She receives a perfect score, but is disappointed to learn that the test had in fact been graded by a computer. A defendant finds that a criminal risk prediction system categorized him as high risk for failure to appear in court, based on the behavior of others like him, despite his having every intention of appearing in court on the appointed date. An automated system locked out a social media user for violating the platform’s policy on acceptable behavior. The user insists that they did nothing wrong, but the platform won’t provide further details nor any appeal process. All of these are automated decision-making or decision support systems that likley feel unfair or unjust. Yet this is a sense of unfairness that is distinct from what we talked about in the first chapter (and which we will return to in the next chapter). It is not about the relative treatment of different groups. Instead, what these questions

2 These three scenarios have something in common: A student is proud of the creative essay she wrote for a standardized test. She receives a perfect score, but is disappointed to learn that the test had in fact been graded by a computer. A defendant finds that a criminal risk prediction system categorized him as high risk for failure to appear in court, based on the behavior of others like him, despite his having every intention of appearing in court on the appointed date. An automated system locked out a social media user for violating the platform’s policy on acceptable behavior. The…

saved by

• Thuraya Ibadbekova

related reading

• The impact of AI errors in a human-in-the-loop processlink.springer.com
• Beyond explainability: justifiability and contestability of Algorithmic Decision Systemsinria.hal.science
• AI and prediction of criminal justice risk: revisiting the COMPAS dataset, experiments, and the problem of algorithmic bias and fairnesslink.springer.com
• Algorithmic decision-making in financial services: economic and normative outcomes in consumer creditlink.springer.com
• The Death of the Legal Subject: How Predictive Algorithms Are (Re)constructing Legal Subjectivityfacctconference.org
• EUR-Lex - 62021CJ0634 - ENeur-lex.europa.eu
• INTERNATIONAL HUMAN RIGHTS LAW AS A FRAMEWORK FOR ALGORITHMIC ACCOUNTABILITY | International & Comparative Law Quarterly | Cambridge Corecambridge.org
• Microsoft Word - 03_Trites Volume 11 issue 2 Final.docxgmj-canadianedition.ca
• Relative notions of fairnessfairmlbook.org
• Toward Idealized Decision Theoryarxiv.org
• Algorithmic Accountabilitydatasociety.net
• The Limits of Dataissues.org