The Public Safety Assessment: A Re-Validation and Assessment of Predictive Utility and Differential Prediction by Race and Gender in Kentucky

Abstract

In this paper, we assess the predictive validity and differential prediction by race and gender of one pretrial risk assessment, the Public Safety Assessment (PSA). The PSA was developed with support from the Laura and John Arnold Foundation (LJAF) to reduce the burden placed on vulnerable populations at the frontend of the criminal justice system. The growing and disparate use of incarceration is one of the most pressing social issues facing the U.S. Although it has received less attention, pretrial populations are a large and growing contributor of mass incarceration. The pretrial phase is often said to be the most consequential in the criminalizing process because it is related a higher likelihood of conviction, longer terms of incarceration, and has the potential to destabilize families. Recognizing the inherent challenges in pretrial release decisions, there has been increased development and use of pretrial risk assessments. Pretrial risk assessments are developed to identify the likelihood that defendants will remain crime free and that they will return to court. There have been several critiques of risk assessments, but none have assessed differential validity or prediction using pretrial outcomes. Using a statewide dataset from Kentucky (n = 164,597) we found the PSA to have predictive validity measures in line with what are generally accepted within the criminal justice field. We applied a regression modeling approach commonly used to assess bias in test instruments (e.g., cognitive and employment testing), and found some instances of differential prediction by race. These differences suggest that the PSA scores to predict failure to appear (FTA) are moderated by race, with no significant differences found for new crimes and new violent crimes between black and white defendants. The findings show differential prediction for new violent criminal arrests between male and female defendants, similar to what was found by Skeem et al. (2016). In the end, we point to data limitations that weaken external validity, point to areas for future research, and suggest that risk assessments are not silver bullets, but rather decision-making tools that require ongoing refinement.