Accuracy and Fairness for Juvenile Justice Risk Assessments

Abstract

Risk assessment algorithms used in criminal justice settings are often said to introduce “bias.” But such charges can conflate an algorithm's performance with bias in the data used to train the algorithm with bias in the actions undertaken with an algorithm's output. In this article, algorithms themselves are the focus. Tradeoffs between different kinds of fairness and between fairness and accuracy are illustrated using an algorithmic application to juvenile justice data. Given potential bias in training data, can risk assessment algorithms improve fairness and, if so, with what consequences for accuracy? Although statisticians and computer scientists can document the tradeoffs, they cannot provide technical solutions that satisfy all fairness and accuracy objectives. In the end, it falls to stakeholders to do the required balancing using legal and legislative procedures, just as it always has.