Abstract
BackgroundThe 2021 NIJ recidivism forecasting challenge asks participants to construct predictive models of recidivism while balancing false positive rates across groups of Black and white individuals through a multiplicative fairness score. We investigate the performance of several models for forecasting 1-year recidivism and optimizing the NIJ multiplicative fairness metric.MethodsWe consider standard linear and logistic regression, a penalized regression that optimizes a convex surrogate loss (that we show has an analytical solution), two post-processing techniques, linear regression with re-balanced data, a black-box general purpose optimizer applied directly to the NIJ metric and a gradient boosting machine learning approach.ResultsFor the set of models investigated, we find that a simple heuristic of truncating scores at the decision threshold (thus predicting no recidivism across the data) yields as good or better NIJ fairness scores on held out data compared to other, more sophisticated approaches. We also find that when the cutoff is further away from the base rate of recidivism, as is the case in the competition where the base rate is 0.29 and the cutoff is 0.5, then simply optimizing the mean square error gives nearly optimal NIJ fairness metric solutions.ConclusionsThe multiplicative metric in the 2021 NIJ recidivism forecasting competition encourages solutions that simply optimize MSE and/or use truncation, therefore yielding trivial solutions that forecast no one will recidivate.
Highlights
The 2021 NIJ recidivism forecasting challenge is a competition hosted by the National Institute of Justice with the aim to “increase public safety and improve the fair administration of justice across the United States”1
Challenge participants are tasked with constructing a predictive model of 1, 2, and 3 year recidivism upon release from prison based on variables such as age
Mohler and Porter Crime Sci (2021) 10:17 of categories aimed at balancing low MSE while reducing the difference of false positive rates between groups of Black and white individuals in the data
Summary
We consider several alternative linear regression models for optimizing the NIJFM. All linear models will be of the form, pi = Xit θ ,. We use a simple shrinkage method where, for Black individuals above the decision boundary cutoff (0.5 or greater for the NIJ competition), we subtract a constant value ǫ from their scores and take the max of that value and the cutoff minus .0001.3 We choose the value of ǫ that optimizes the NIJFM on the training data. We refer to this method as linear regression with shrinkage. The covariates we use to construct our models include: gender (sex), race (Black or white), an indicator for being gang affiliated, age at release from prison, years spent in prison, total prior arrests, total prior convictions, education level, and number of dependents
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
