When is automated decision making legitimate?

A student is proud of the creative essay she wrote for a standardized test. She receives a perfect score, but is disappointed to learn that the test had in fact been graded by a computer. A defendant finds that a criminal risk prediction system categorized him as high risk for failure to appear in court, based on the behavior of others like him, despite his having every intention of appearing in court on the appointed date. An automated system locked out a social media user for violating the platform’s policy on acceptable behavior. The user insists that they did nothing wrong, but the platform won’t provide further details nor any appeal process. All of these are automated decision-making or decision support systems that likley feel unfair or unjust. Yet this is a sense of unfairness that is distinct from what we talked about in the first chapter (and which we will return to in the next chapter). It is not about the relative treatment of different groups. Instead, what these questions

2 These three scenarios have something in common: A student is proud of the creative essay she wrote for a standardized test. She receives a perfect score, but is disappointed to learn that the test had in fact been graded by a computer. A defendant finds that a criminal risk prediction system categorized him as high risk for failure to appear in court, based on the behavior of others like him, despite his having every intention of appearing in court on the appointed date. An automated system locked out a social media user for violating the platform’s policy on acceptable behavior. The…

saved by

• Thuraya Ibadbekova

related reading

• The impact of AI errors in a human-in-the-loop processlink.springer.com
• Algorithmic decision-making in financial services: economic and normative outcomes in consumer creditlink.springer.com
• The Death of the Legal Subject: How Predictive Algorithms Are (Re)constructing Legal Subjectivityfacctconference.org
• AI and prediction of criminal justice risk: revisiting the COMPAS dataset, experiments, and the problem of algorithmic bias and fairnesslink.springer.com
• Beyond explainability: justifiability and contestability of Algorithmic Decision Systemsinria.hal.science
• Relative notions of fairnessfairmlbook.org
• Algorithmic Accountabilitydatasociety.net
• Microsoft Word - 03_Trites Volume 11 issue 2 Final.docxgmj-canadianedition.ca
• Can we safely automate alignment research? - Joe Carlsmithjoecarlsmith.com
• Why AI alignment could be hard with modern deep learningcold-takes.com
• The Limits of Dataissues.org
• A Course in Machine Learningciml.info