Pretrial Risk Assessment Research Articles

Towards an evidence-based pretrial risk assessment in Mexican juvenile offenders: A systematic review of relevant instruments using COSMIN guidelines

The Mexican Comprehensive Criminal Justice System for Adolescents (Sistema Integral de Justicia Penal para Adolescentes) is in urgent need of validated tools to help diminish the likelihood of pretrial failure, (that is, when juveniles interfere in one way or another with the course of the criminal process before the trial stage). To this end, this article aims to evaluate the measurement properties of relevant instruments to guide and support pretrial risk assessment in Mexican juvenile offenders. Firstly, a systematic review was conducted in PubMed, metasearch engines (DGB-UNAM and Google Scholar), and databases using the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines. As it was found that no validated pretrial risks assessment instruments had been published earlier in Mexico, we present a proposal based on a preliminary selection of five instruments suitable for pretrial risks assessment taking both analysis and theory into account. Since this is the first systematic review in the field, results provide evidence for developing pretrial risk tools to aid decision-making in the juvenile offenders sector in Mexico.

Testing Predictive Biases at the Intersection of Race-Ethnicity and Sex: A Multi-Site Evaluation of a Pretrial Risk Assessment Tool

Pretrial assessment instruments provide courtroom actors with information about the likelihood that someone who has been charged with a criminal offense would engage in criminal behavior if released prior to trial. Although prior research supports the ability of pretrial instruments to predict pretrial outcomes, there are concerns that pretrial instruments may inadvertently exacerbate racial-ethnic and sex disparities found in the larger criminal legal system. In the current study, we conduct multi-site predictive bias tests of a widely used pretrial instrument—the Public Safety Assessment (PSA)—across six racial-ethnic and sex groups (i.e., White males, White females, Black males, Black females, Hispanic males, and Hispanic females). Study results support the PSA as a valid and consistent predictor of failure to appear, new criminal activity, and new violent criminal activity across these six racial-ethnic and sex groups. These findings support the use of the PSA and identify several areas for future research.

An Item-Based Revalidation of the Florida Pretrial Risk Assessment Instrument

ABSTRACT Scholars and practitioners continue to examine the growing pretrial detainee population. These individuals have not been convicted of a crime, and yet they sit in detention, taxing jail facilities and their limited budgets. Pretrial risk assessment tools have been created to identify individuals who can be released prior to trial, without jeopardizing community safety. The current study revalidates items included in the Florida Pretrial Risk Assessment Instrument. Results suggest that most individuals on pretrial release do not commit new offenses or fail to appear for future court hearings. Similarities and differences between the original study and the revalidation are noted. The inclusion of assessing “community ties” is also explored.

Pretrial Risk Assessment on the Ground: Algorithms, Judgments, Meaning, and Policy

Pretrial Risk Assessment on the Ground: Algorithms, Judgments, Meaning, and Policy

Pretrial Decision-Making Matrices: The Role of Risk and Charge Weighting in Risk Assessment–Guided Decisions

Pretrial risk assessments are used in local pretrial settings to inform release and supervision decisions. Yet, there is little research on the implementation of pretrial risk assessments in practice. We examined the characteristics of pretrial decision-making matrices in a statewide sample of counties using the same risk assessment tool. We procured pretrial decision-making matrices from 29 Indiana counties that had received or were in the process of receiving pretrial certification (88.6% response rate). Using a structured coding protocol, we found matrices shared common features but differed in their structure and available release and supervision options. Matrices weighted index charge information more heavily than risk assessment information, suggesting risk assessments likely explain between 20% and 36% of variability in decisions given typical rates of judicial adherence. Local decision-makers should be aware that structuring matrices to weight charge information more heavily than risk assessment information may limit the risk management potential of these tools.

Pretrial Risk Assessment, Release Recommendations, and Racial Bias

We examined how the presentation of risk assessment results and the race of the person charged affected pretrial court actors’ recommendations to release a person with or without conditions. A sample of 246 pretrial court actors read vignettes that varied risk framing (success, failure), risk format (probability, frequency), risk level (low, high), and race of the person charged (Black, White). Pretrial release recommendations did not differ as a function of framing or format overall or by race. Pretrial court actors were more likely to recommend release with conditions compared with release without conditions in the high- versus low-risk groups when the person was White but not when the person was Black. Findings fail to support changes to the presentation of pretrial risk assessment results as strategies to enhance their impact on release recommendations and underscore the need for efforts to ensure equitable application of risk assessment results for people of color.

Testing Predictive Biases at the Intersection of Race-Ethnicity and Sex: A Multi-Site Evaluation of a Pretrial Risk Assessment Tool

Testing Predictive Biases at the Intersection of Race-Ethnicity and Sex: A Multi-Site Evaluation of a Pretrial Risk Assessment Tool

The Impact of Overbooking on a Pre-Trial Risk Assessment Tool

The Impact of Overbooking on a Pre-Trial Risk Assessment Tool

“If it didn’t happen, why would I change my decision?”: How Judges Respond to Counterfactual Explanations for the Public Safety Assessment

Many researchers and policymakers have expressed excitement about algorithmic explanations enabling more fair and responsible decision-making. However, recent experimental studies have found that explanations do not always improve human use of algorithmic advice. In this study, we shed light on how people interpret and respond to counterfactual explanations (CFEs)---explanations that show how a model's output would change with marginal changes to its input(s)---in the context of pretrial risk assessment instruments (PRAIs). We ran think-aloud trials with eight sitting U.S. state court judges, providing them with recommendations from a PRAI that includes CFEs. We found that the CFEs did not alter the judges' decisions. At first, judges misinterpreted the counterfactuals as real---rather than hypothetical---changes to defendants. Once judges understood what the counterfactuals meant, they ignored them, stating their role is only to make decisions regarding the actual defendant in question. The judges also expressed a mix of reasons for ignoring or following the advice of the PRAI without CFEs. These results add to the literature detailing the unexpected ways in which people respond to algorithms and explanations. They also highlight new challenges associated with improving human-algorithm collaborations through explanations.

Comparing the relationships between money bail, pretrial risk scores, and pretrial outcomes.

There has been much discussion around the use of both money bail and pretrial risk assessment instruments. We examine how bail and risk scores compare in terms of their associations with failure to appear in court and rearrest during the pretrial period. Our research questions included whether bail and risk scores differed between people who did and did not experience pretrial outcomes and whether pretrial scores were associated with outcomes when controlling for bail and other relevant covariates. To examine these associations, we drew a sample of 492 people (33% women; 60% Black) booked into county jail in a jurisdiction not yet using a pretrial risk assessment instrument to inform release decisions. We completed the Public Safety Assessment (PSA) for this sample and collected data on failure-to-appear and rearrest incidents for 1 year following initial bookings. We examined the associations between bail amount, failure to appear, and rearrest and between PSA subscale scores, failure to appear, and rearrest. Bail amount was not associated with either failure to appear or rearrest. People who failed to appear or were rearrested had higher bail amounts, on average, than people who did not. In contrast, PSA subscale scores were significantly associated with outcomes in the expected direction. Our findings do not support the use of money bail for ensuring that people return to court and avoid rearrest. Instead, our findings suggest that using pretrial risk assessment, instruments could result in more accurate and appropriate release decisions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Differential Prediction by Race in IRAS-PAT Assessments: An Application of Debiasing Strategies

There remain serious concerns about the potential for pretrial risk assessments to exacerbate racial disparities. Yet, current evidence on differential prediction in pretrial risk assessments is limited. The present investigation tests for differential prediction by race as an indication of bias in Indiana Risk Assessment System–Pretrial Assessment Tool (IRAS-PAT) assessments. Using pooled data drawn from a five-county IRAS-PAT validation, which included 689 Black and 2,850 White defendants, we primarily used a hierarchical regression-based approach to test between-group differences in the slopes of regression lines. Where slope-based bias was present, differential prediction was reevaluated once algorithmic corrections were applied. Findings showed IRAS-PAT assessments produced less accurate predictions of pretrial misconduct for Black defendants relative to White defendants. Only one debiasing strategy—which accounted for item-level differences across subgroups—corrected differential prediction. Debiasing strategies can mitigate differential prediction but may have limited utility for local jurisdictions under current legal frameworks.

Pretrial risk assessment instruments in the US criminal justice system—what lessons can be learned for the European Union

Abstract This article explores the possibilities of using artificial intelligence (AI) in the European judicial area with a special emphasis on risk assessment instruments and the justification of their use in determining the likelihood that a defendant at large will seek to thwart the goal of criminal proceedings. In this regard, experiences in the application of various risk assessment instruments in US judicial practice are first analysed. Then the European normative framework established at the level of the Council of Europe and European Union (EU) law is considered with special emphasis on the ‘Proposal for a Regulation of the European Parliament and of the Council laying down harmonized rules on artificial intelligence’ as a key normative act that will determine the long-term preconditions for the application of the risk assessment instruments in the criminal proceedings of EU Member States. Finally, an attempt is made to give answers to some specific questions that arose from the implementation of AI in EU criminal justice systems taking into account the presumption of innocence, the right to an objective and impartial judge and the principle of proportionality.

Pretrial risk assessment instruments in practice: The role of judicial discretion in pretrial reform

AbstractResearch SummaryWe explored the extent to which the implementation of a pretrial risk assessment instrument (PRAI) corresponded to changes in the pretrial processing of defendants using multiple administrative data sources from a large county in the southeastern United States. Our findings revealed little evidence of reductions in detention lengths or increases in the use of nonfinancial forms of release following the tool's adoption. This was largely attributable to the exercise of judicial discretion, as judges frequently departed from the tool's recommendation using alternatives that were more punitive and often included financial conditions—particularly for Black and Latino defendants. Furthermore, the exercise of discretion was linked to increased rates of pretrial failure.Policy ImplicationsPRAIs were adopted on a massive scale with the understanding that they are evidence‐based and geared toward efficiently and equitably reducing pretrial populations; however, we are lacking the evaluative work to determine their impacts. Our findings suggest that PRAIs may not only undermine reform efforts, but may worsen disparities, if communities fail to complete the up‐front work of discussing their expectations for pretrial decision making, including the conditions under which financial constraints may be justifiable.

Differential prediction and disparate impact of pretrial risk assessments in practice: a multi-site evaluation

Differential prediction and disparate impact of pretrial risk assessments in practice: a multi-site evaluation

Evaluating Fairness of Algorithmic Risk Assessment Instruments: The Problem With Forcing Dichotomies

Researchers and stakeholders have developed many definitions to evaluate whether algorithmic pretrial risk assessment instruments are fair in terms of their error and accuracy. Error and accuracy are often operationalized using three sets of indicators: false-positive and false-negative percentages, false-positive and false-negative rates, and positive and negative predictive value. To calculate these indicators, a threshold must be set, and continuous risk scores must be dichotomized. We provide a data-driven examination of these three sets of indicators using data from three studies on the most widely used algorithmic pretrial risk assessment instruments: the Public Safety Assessment, the Virginia Pretrial Risk Assessment Instrument, and the Federal Pretrial Risk Assessment. Overall, our findings highlight how conclusions regarding fairness are affected by the limitations of these indicators. Future work should move toward examining whether there are biases in how the risk assessment scores are used to inform decision-making.

The Empirical Case for Pretrial Risk Assessment Instruments

Pretrial risk assessment instruments are used in many jurisdictions to inform decisions regarding pretrial release and conditions. Many are concerned that the use of pretrial risk assessment instruments may be contributing to worsened, not improved, pretrial outcomes, including increased rates of pretrial detention and exacerbated racial disparities in pretrial decisions. These concerns have led prominent organizations to reverse their position on the role of pretrial risk assessment instruments in pretrial system change. Reforms that centered on their use have been rolled back or have failed to be implemented in the first place. However, the scientific evidence behind these concerns is lacking. Instead, the findings of rigorous research show that the results of pretrial risk assessment instruments demonstrate good accuracy in predicting new criminal activity, including violent crime, during the pretrial period, even when there are differences between groups defined by race and ethnicity. Furthermore, the scientific evidence suggests they can be an effective strategy to help achieve pretrial system change, including reducing pretrial detention for people of color and white people, alike, when their results are actually used to inform decision-making. In this article, we review the scientific evidence in relation to three primary critiques of pretrial risk assessment instruments, namely, that their results have poor accuracy and are racially biased and that their use increases pretrial detention rates. We also provide recommendations for addressing these critiques to ensure that their use supports, rather than detracts from, the goals of pretrial reform and articulates an agenda for future research.

Pretrial risk assessment validation research: Range restriction and attenuation of predictive validity estimates.

Pretrial risk assessment validation research: Range restriction and attenuation of predictive validity estimates.

Use of Risk Assessments in Pretrial Supervision Decision-Making and Associated Outcomes

Despite the growing use of both pretrial risk assessment and supervision as pretrial reform strategies, there has been limited investigation on the effectiveness of risk assessment-informed pretrial supervision. We conducted a multi-site, retrospective investigation in 1,505 pretrial defendants from four local jurisdictions to examine risk assessment-informed pretrial supervision decisions and associated pretrial misconduct outcomes. Our findings showed pretrial supervision decisions were generally consistent with structured guidelines and defendant risk classifications. Use of bond and electronic monitoring had little impact on pretrial misconduct. However, more frequent pretrial monitoring was associated with higher rates of pretrial misconduct across all risk levels. Reducing supervision conditions and monitoring for low-risk defendants, in particular, may help reduce rates of misconduct in pretrial populations.

When Politicization Stops Algorithms in Criminal Justice

Abstract The present paper sheds light on the conditions for successfully implementing pretrial risk assessment tools by looking at barriers to their implementation and, specifically, political factors that can derail this process. Drawing on theoretical assumptions about the role of agenda-setting and public discourse in policy change, we examine three US states where the implementation of pretrial tools was halted or heavily restricted. Our findings suggest that a politicization of these tools plays a central role for thwarting their implementation. This has less to do with their technical properties or performance. Rather, through reaching high-level politics, those tools are publicly linked to concerns about opacity, fairness and public safety such that it simply becomes politically safer for politicians to stick with the status quo.

Evaluating the evidence in algorithmic evidence-based decision-making: the case of US pretrial risk assessment tools

ABSTRACT Algorithmic decision-making (ADM) promises to strengthen evidence-based decisions, particularly to better manage risks in various domains. Its use also extends to the criminal justice system where algorithmic risk assessments potentially provide very valuable evidence that can inform highly sensitive decisions. Yet, such algorithmic tools also introduce intricate problems that are tied to the fundamental question of exactly what kind and what quality of evidence they offer. This paper illustrates this problem based on a comparison of pretrial risk assessments that have been implemented statewide in the USA. The authors highlight the empirical variation in the construction, evaluation and documentation of these tools to carve out the considerable discretion involved along these dimensions. They also point to further possible ways of looking at the performance of these tools and show why evaluating the quality of the evidence delivered by algorithmic risk assessments is a far from straightforward affair.