Actuarial Risk Assessment Instruments Research Articles

Machine Learning and Risk Assessment: Random Forest Does Not Outperform Logistic Regression in the Prediction of Sexual Recidivism.

Although many studies supported the use of actuarial risk assessment instruments (ARAIs) because they outperformed unstructured judgments, it remains an ongoing challenge to seek potentials for improvement of their predictive performance. Machine learning (ML) algorithms, like random forests, are able to detect patterns in data useful for prediction purposes without explicitly programming them (e.g., by considering nonlinear effects between risk factors and the criterion). Therefore, the current study aims to compare conventional logistic regression analyses with the random forest algorithm on a sample of N = 511 adult male individuals convicted of sexual offenses. Data were collected at the Federal Evaluation Center for Violent and Sexual Offenders in Austria within a prospective-longitudinal research design and participants were followed-up for an average of M = 8.2 years. The Static-99, containing static risk factors, and the Stable-2007, containing stable dynamic risk factors, were included as predictors. The results demonstrated no superior predictive performance of the random forest compared with logistic regression; furthermore, methods of interpretable ML did not point to any robust nonlinear effects. Altogether, results supported the statistical use of logistic regression for the development and clinical application of ARAIs.

Die Vorhersage von Lockerungsmissbräuchen und intramuralen Regelverstößen mittels aktuarischer Prognoseinstrumente – eine retrospektive Validierungsstudie der OGRS 3 und des SVG-5

ZusammenfassungKriminalprognostische Verfahren sind unerlässlicher Bestandteil der Beurteilung von Rückfallwahrscheinlichkeiten im Zuge von Entlassungsentscheidungen. Ihre Eignung zur Vorhersage von Lockerungsmissbräuchen und intramuralen Regelverstößen wurde hingegen bislang kaum untersucht. Die vorliegende Studie prüft die prädiktive Validität der 3. Version der Offender Group Reconviction Scale (OGRS 3) und des Screeninginstrument zur Vorhersage des Gewaltrisikos (SVG-5) an einer Stichprobe von 200 Insassen der Justizvollzugsanstalt Frankenthal. Darüber hinaus werden weitere potenzielle Prädiktoren explorativ untersucht. Für die Prognoseinstrumente ergeben sich überwiegend geringe bis maximal moderate Effektstärken (OGRS 3: Area Under the Curve (AUC) = 0,522 bis 0,556 und SVG-5: AUC = 0,561 bis 0,653). Die vorliegenden Ergebnisse sollten aufgrund methodischer Limitationen allerdings zurückhaltend interpretiert werden. Sie können jedoch als Ausgangspunkt und Grundlage für zukünftige Forschung in diesem Bereich verwendet werden, da die Resultate nahelegen, dass auch Lockerungsmissbräuche und intramurales Fehlverhalten grundsätzlich vorhergesagt werden können.

A comparison of the predictive accuracy of structured and unstructured risk assessment methods for the prediction of recidivism in individuals convicted of sexual and violent offense.

One of the most commonly replicated results in the research area of recidivism risk assessment is the superiority of structured and standardized prediction methods in comparison to unstructured, subjective, intuitive, or impressionistic clinical judgments. However, the quality of evidence supporting this conclusion is partly still controversially discussed because studies including direct comparisons of the predictive accuracy of unstructured and structured risk assessment methods have been relatively rarely conducted. Therefore, we examined in the present study retrospectively N = 416 expert witness reports written about individuals convicted of violent and/or sexual offenses in Germany between 1999 and 2015. The predictive accuracy of different methodological approaches of risk assessment (subjective clinical [i.e., unstructured clinical judgment; UCJ], structured professional judgment [SPJ], actuarial risk assessment instruments [ARAIs], and combinations of ARAIs-/SPJ-based risk assessments) was compared by analyzing the actual reoffenses according to the Federal Central Register (average follow-up period M = 7.08 years). In accordance with previously published results, the results indicated a higher predictive accuracy for structured compared to unstructured risk assessment approaches for the prediction of general, violent, and sexual recidivism. Taken together, the findings underline the limited accuracy of UCJs and provided further support for the use of structured and standardized risk assessment procedures in the area of crime and delinquency. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The predictive and incremental validity of ADHD beyond the VRAG-R in a high-risk sample of young offenders

The VRAG-R is a well-established actuarial risk-assessment instrument, which was originally developed for assessing violent recidivism risk in adult male offenders. Whether or not the VRAG-R can also predict violent recidivism in young offenders is unclear so far. In the emergence of juvenile offending, attention-deficit/hyperactivity disorder (ADHD) seems to be of major importance suggesting that it could be relevant for risk assessment as well. Thus, we examined the predictive accuracy of the VRAG-R in a high-risk sample of N = 106 (M = 18.3 years, SD = 1.8) young offenders and assessed the incremental predictive validity of ADHD symptomatology beyond the VRAG-R. Within a mean follow-up time of M = 13 years (SD = 1.2), n = 65 (62.5%) young offenders recidivated with a violent offense. We found large effect sizes for the prediction of violent and general recidivism and re-incarcerations using the VRAG-R sum scores. Current ADHD symptomatology added incremental predictive validity beyond the VRAG-R sum scores concerning the prediction of general recidivism but not of violent recidivism. The results supported the use of the VRAG-R for predicting violent recidivism in young offenders. Because ADHD symptomatology improves the predictive performance of the VRAG-R regarding general recidivism, we argue that addressing ADHD symptoms more intensively in the juvenile justice system is of particular importance concerning a successful long-term risk management in adolescents and young adults.

The Trouble with Using Risk Assessment Instruments to Quantify the Chance of Future Offending

Risk assessments in carceral settings have proliferated in recent decades and are now prominent in numerous states and regions. A ubiquitous variety is actuarial risk assessment instruments that are used on children and adults to predict their future chance for misconduct (e.g., recidivism) in several vital decision points in carceral processing (e.g., pretrial confinement). These instruments rely on information about past behavior (e.g., criminal history) and an understanding of offending (e.g., antisocial personality) that is thought to be neutral, reliable, and enjoys predictive validity. However, it will be argued that when justice system personnel assess the chance of unwanted behavior in the future, several risk domains are differentially prevalent and more frequently experienced by some groups. Much of this disparity is caused by, or due to, forces external to those being assessed, for instance, inequitable social and economic conditions and inequitable decisions by justice personnel to arrest, charge, or sentence people of color. As such, risk assessment instruments inevitably and disproportionately mark some groups of people as a higher risk to violate rules, conditions, orders, or laws. Consequently, risk assessment instruments systematically disfavor disadvantage, and by inference, favor advantage, leading to the need for a radical shift in the taxonomy of classifying risk for future misconduct.

What Works (or Does Not) in Community Risk Management for Persons Convicted of Sexual Offenses? A Contemporary Perspective.

Contemporary data from the United States show that rates of sexual offending and reoffending have been in steady decline for decades. Nonetheless, nonprofessionals continue to view sexual violence as a community safety issue fraught with risk and uncertainty. The past 30 years have been witness to considerable research and practice in the assessment, treatment, and risk management of persons who have sexually offended. Gains have also been made in regard to prevention and citizen education. Modern day technologies include actuarial risk assessment instruments, measures of criminogenic need and treatment progress, refinements to treatment processes, and the establishment of evidence-based models. Legislative authorities in the United States and elsewhere have also attempted to affect risk in the community with, perhaps, lesser degrees of success. This article reviews current policies and practices, with a specific focus on what happens when offenders are released to the community (e.g., how public policies intended to track offenders and/or restrict their movements can negatively affect community reintegration). Comprehensive approaches to community sexual offender management are examined in addition to suggestions of unique approaches intended to ensure citizen buy-in and engagement.

Assisting decisions in child protection service institutions with the RIC – The Risk Indication in Child sexual abuse

BackgroundThe RIC (Risk Indication in Child sexual abuse) and its screening version (RIC:SV) are actuarial risk assessment instruments, developed at the Austrian Federal Evaluation Centre for Violent and Sexual Offenders and designed for child protection services to assess the likelihood of sexual recidivism in male contact child sexual abusers who still or again live within a family including children. ObjectiveThe RIC was designed to require a minimum of forensic information, with the RIC:SV completely waiving such information. ParticipantsNine factors related to sexual recidivism could be identified by analyzing five-year follow-up data of N = 324 male contact child sexual abusers. SettingThe data was collected retrospectively from files between the years 2002 and 2011. MethodChi-Square Tests and ROC-analyses were calculated. ResultsThe RIC and the RIC:SV were found to significantly predict sexual recidivism with AUC values of .84 (RIC) and .78 (RIC:SV). The items of the RIC are: (a) offender has never had a live-in relationship, (b) unstable family background of the offender, (c) impulsivity of the offender, (d) previous psychiatric treatment(s) of the offender, (e) offender has prior prison sentence/s, (f) problems of the offender to accept rules and norms, (g) offender was a stranger to his former victim(s), (h) offender had at least one male victim, and (i) offender has ever committed an extrafamilial contact child sexual abuse offense. ConclusionThe RIC and the RIC:SV need no extensive training to be used. The results may help CPS workers to justify risk related interventions.

Actuarial risk assessment at sentencing: Potential consequences for mass incarceration and legitimacy.

The growing utilization of actuarial risk assessment instruments (RAIs) in the American criminal justice system may potentially lead to more restrained, better-targeted uses of incarceration. However, critics have raised a variety of concerns regarding RAIs and suggested, among other things, that their incorporation into court procedures may tend to dehumanize defendants and exacerbate, rather than alleviate, mass incarceration. In response to such concerns, this article offers recommendations for policy and practice that may help an increasingly risk-focused criminal justice system to achieve decarceration goals - and to do so without undermining its own legitimacy in the eyes of defendants or the public at large. These recommendations are aimed particularly at judges, in their roles as sentencers and makers of courtroom and courthouse policy.

Safety and Risk Assessment in Child Welfare: A Reliability Study Using Multiple Measures

ObjectivesIn the Netherlands, the Actuarial Risk Assessment Instrument for Youth Protection (ARIJ) is a widely used safety and risk assessment instrument in child welfare, although little is known about its reliability. Therefore, this study aimed to determine the reliability of the ARIJ by examining the inter- and intrarater reliability.MethodsFor determining interrater reliability, professionals of two Dutch agencies (child and family support, n= 39 & child protection, n = 24) and master students (n= 65) each rated a random selection of 4 out of 24 vignettes. The vignettes were based on actual cases that were handled by the two agencies. For determining intrarater reliability, the professionals rated four vignettes twice with an interval of at least 3 months. Three reliability measures were calculated for each of the three samples: percent agreement, Krippendorff’s alpha, and Gwet’s Agreement Coefficient.ResultsOverall, the items and outcome of the safety assessment instrument showed a moderate or higher than moderate interrater reliability, and a substantial to almost perfect intrarater reliability. In general, the risk assessment items showed a moderate interrater and a substantial-to-high intrarater reliability. The risk assessment outcome had a near perfect interrater reliability and a substantial to almost perfect intrarater reliability.ConclusionsThe outcome of both the safety and risk assessment of the ARIJ proved to be reliable and justifies the use of the ARIJ in the Dutch child welfare by professionals with different levels of experience.

Primer on the Contribution of Crime Scene Behavior to the Forensic Assessment of Sexual Offenders

Abstract. Over the last decades several attempts in developing incrementally valid risk indicators above and beyond standard actuarial and dynamic risk assessment instruments have been undertaken without much success. The current review will summarize current developments regarding the validity of detailed crime scene analysis for forensic assessments of sexual offenders. To this end, this overview will focus on two issues: First, we will discuss the issue of sexual offender risk assessment based on crime scene information. Second, we will outline how crime scene behavior contributes to the assessment of sexual interest in children. In each section we will introduce the reader to new approaches and instruments for risk assessment and diagnosing pedophilic sexual interest based on crime scene information, respectively, and discuss their validity. Finally, we will discuss the implications of these recent developments for risk assessment, risk management, offender treatment, and future research.

Actuarial risk assessment of sexual offenders: The psychometric properties of the Sex Offender Risk Appraisal Guide (SORAG).

The Sex Offender Risk Appraisal Guide (SORAG) is one of the most commonly used actuarial risk assessment instruments for sexual offenders. The aims of the present field study were to examine the predictive validity of the German version of the SORAG and its individual items for different offender subgroups and recidivism criteria in sexual offenders released from the Austrian Prison System (N = 1,104; average follow-up period M = 6.48 years) within a prospective-longitudinal research design. For the prediction of violent recidivism the German version of the SORAG yielded an effect size of AUC = .74 (p < .001, 95% CI = .70-.78). The predictive accuracy for general and violent recidivism was slightly higher than for general sexual and sexual hands-on recidivism. The effect sizes were found to be higher for the child molester sample than for rapists. However, the differences were significant only for general recidivism (z = 2.48, p = .001). Further analyses exhibited the SORAG to have incremental predictive validity beyond the VRAG and the PCL-R, and to remain the only significant predictor for violent recidivism once all 3 instruments were forced into a combined regression model. Twelve out of the 14 SORAG items were found to have a significant positive relationship with violent recidivism. The comparison of the relative and absolute risk indices between the Austrian and the Canadian samples showed that the normative data distribution yielded more (absolute risk indices) or less (relative risk indices) meaningful differences between the 2 countries. (PsycINFO Database Record

The Influence of Risk Assessment Instrument Scores on Evaluators’ Risk Opinions and Sexual Offender Containment Recommendations

In Colorado, evaluators conducting sex offender risk assessments are required to assess 17 risk factors specified by the state’s Sex Offender Management Board (SOMB), in addition to scoring actuarial risk assessment instruments. This study examined the association between instrument scores, the 17 SOMB risk factors, and evaluator opinions concerning risk and need for containment in 302 Colorado cases. Evaluators’ ratings of risk indicated by noninstrument factors were often higher than their ratings of risk indicated by instrument results, but only their ratings of noninstrument factors were independently predictive of containment recommendations. Several of the most influential noninstrument factors (e.g., denial, treatment motivation) have been described by researchers as potentially misleading because they are not predictive of future offending. Findings highlight the need for more studies examining the validity of what risk assessment evaluators actually do, as opposed to what researchers think they should do.

The Adam Walsh Act: An Examination of Sex Offender Risk Classification Systems.

This study was designed to compare the Adam Walsh Act (AWA) classification tiers with actuarial risk assessment instruments and existing state classification schemes in their respective abilities to identify sex offenders at high risk to re-offend. Data from 1,789 adult sex offenders released from prison in four states were collected (Minnesota, New Jersey, Florida, and South Carolina). On average, the sexual recidivism rate was approximately 5% at 5 years and 10% at 10 years. AWA Tier 2 offenders had higher Static-99R scores and higher recidivism rates than Tier 3 offenders, and in Florida, these inverse correlations were statistically significant. Actuarial measures and existing state tier systems, in contrast, did a better job of identifying high-risk offenders and recidivists. As well, we examined the distribution of risk assessment scores within and across tier categories, finding that a majority of sex offenders fall into AWA Tier 3, but more than half score low or moderately low on the Static-99R. The results indicate that the AWA sex offender classification scheme is a poor indicator of relative risk and is likely to result in a system that is less effective in protecting the public than those currently implemented in the states studied.

Acquaintance Rape: Applying Crime Scene Analysis to the Prediction of Sexual Recidivism.

The aim of the current study was to enhance the assessment and predictive accuracy of risk assessments for sexual offenders by utilizing detailed crime scene analysis (CSA). CSA was conducted on a sample of 247 male acquaintance rapists from Berlin (Germany) using a nonmetric, multidimensional scaling (MDS) Behavioral Thematic Analysis (BTA) approach. The age of the offenders at the time of the index offense ranged from 14 to 64 years (M = 32.3; SD = 11.4). The BTA procedure revealed three behavioral themes of hostility, criminality, and pseudo-intimacy, consistent with previous CSA research on stranger rape. The construct validity of the three themes was demonstrated through correlational analyses with known sexual offending measures and criminal histories. The themes of hostility and pseudo-intimacy were significant predictors of sexual recidivism. In addition, the pseudo-intimacy theme led to a significant increase in the incremental validity of the Static-99 actuarial risk assessment instrument for the prediction of sexual recidivism. The results indicate the potential utility and validity of crime scene behaviors in the applied risk assessment of sexual offenders.

Validity and reliability of the VRAG in a forensic psychiatric medium security population in Flanders

ABSTRACTSince the development of the Violence Risk Appraisal Guide (VRAG), one of the most widely used actuarial risk assessment instruments, numerous replication studies have shown its usefulness in predicting (violent) recidivism among various offender populations. It is not clear, however, whether these findings can be generalized to forensic psychiatric patients admitted to a medium security forensic psychiatric unit in the Flemish part of Belgium. Therefore, the main aim of this study was to test the predictive validity and reliability of the VRAG in a sample of 191 Flemish medium security patients. The mean follow-up period was 2.4 years. Contrary to the expectations, the VRAG was unable to significantly predict any kind of outcome. Possible explanations are discussed and further research with the VRAG in Flanders, and its recently revised version, is recommended.

Evidence for risk estimate precision: implications for individual risk communication.

Actuarial risk assessment instruments using well-established predictor variables measured at the individual level (e.g., age, criminal history, psychopathy) discriminate well between recidivists and non-recidivists across diverse samples. Data indicating the relative risk of recidivism can inform policy decisions about allocating resources according to risk within a correctional system, consistent with the first of the risk-need-responsivity (RNR) principles. Evidence for the precision of absolute risk as applied to an individual based on scores from many samples, however, has proven challenging. In this paper, we present a study examining the association of actuarial risk estimate precision with sample size using the Post Conviction Risk Assessment (PCRA; Lowenkamp et al., 2013), in samples of up to 26,642 offenders. Results indicate that the precision of individual estimates can be demonstrated with sufficient sample size. We believe that the implications of absolute risk for the communication of an individual offender's risks have been poorly understood. We argue that the purpose of individual-level risk communication is to ensure the effective application of policy, which requires matching a new case to aggregate data. We illustrate how an offender's risk might thus be communicated, and conclude that this function is distinct from management of an individual's criminogenic needs and identification of effective and suitable treatments.

A Decoupled System: Federal Criminal Justice and the Structural Limits of Transformation

The United States federal criminal justice system is changing. Actuarial risk assessment instruments and evidence-based practices play increasingly important roles; federal reentry court programs have been implemented across the country. Yet, while promising, these developments may not be enough to stem the growth and costs of federal criminal justice. The highly politicized nature of crime and punishment may limit the potential for change. Even within the federal criminal justice system, the decoupled nature of bureaucracies, in which stakeholders make decisions for which they are not financially responsible, makes meaningful change problematic. States, however, have demonstrated that structural changes can foster efficient use of resources and improve fiscal stewardship. A number of statutory, structural, and procedural modifications could help to reorganize a continuum of fragmented bureaucracies into a cohesive federal reentry-centered system.

Replicating the violence risk appraisal guide: a total forensic cohort study.

IntroductionThe performance of violence risk assessment instruments can be primarily investigated by analysing two psychometric properties: discrimination and calibration. Although many studies have examined the discrimination capacity of the Violence Risk Appraisal Guide (VRAG) and other actuarial risk assessment tools, few have evaluated how well calibrated these instruments are. The aim of the present investigation was to replicate the development study of the VRAG in Europe including measurements of discrimination and calibration.MethodUsing a prospective study design, we assessed a total cohort of violent offenders in the Zurich Canton of Switzerland using the VRAG prior to discharge from prisons, secure facilities, and outpatient clinics. Assessors adhered strictly to the assessment protocol set out in the instrument’s manual. After controlling for attrition, 206 offenders were followed in the community for a fixed period of 7 years. We used charges and convictions for subsequent violent offenses as the outcomes. Receiver operating characteristic analysis was conducted to measure discrimination, and Sanders’ decomposition of the Brier score as well as Bayesian credible intervals were calculated to measure calibration.ResultsThe discrimination of the VRAG’s risk bins was modest (area under the curve = 0.72, 95% CI = 0.63–0.81, p<0.05). However, the calibration of the tool was poor, with Sanders’ calibration score suggesting an average assessment error of 21% in the probabilistic estimates associated with each bin. The Bayesian credible intervals revealed that in five out of nine risk bins the intervals did not contain the expected risk rates.DiscussionMeasurement of the calibration validity of risk assessment instruments needs to be improved, as has been done with respect to discrimination. Additional replication studies that focus on the calibration of actuarial risk assessment instruments are needed. Meanwhile, we recommend caution when using the VRAG probabilistic risk estimates in practice.

Family Risk Assessment: Characteristics of Families With Child Abuse Notifications in Australia

Valid and reliable assessments are required by Courts to assist in the determination of child-at-risk findings, and in the allocation of abused children into care. This study reports on the construct validation of an actuarial risk assessment instrument: the Family Risk of Abuse and Neglect (FRAAN). One hundred and fifty expert reports were reviewed and two superordinate constructs representing history of child abuse and appropriate parenting skills respectively were extracted from the reports. The results revealed that high FRAAN scores were associated with a history of investigations for child abuse, but there was no relationship between FRAAN scores and the second construct. The results support the validity of the FRAAN for assessing risk factors associated with child abuse.

From Group Data to Useful Probabilities: The Relevance of Actuarial Risk Assessment in Individual Instances

Probability plays a ubiquitous role in decision-making through a process in which we use data from groups of past outcomes to make inferences about new situations. Yet in recent years, many forensic mental health professionals have become persuaded that overly wide confidence intervals render actuarial risk assessment instruments virtually useless in individual assessments. If this were true, the mathematical properties of probabilistic judgments would preclude forensic clinicians from applying group-based findings about risk to individuals. As a consequence, actuarially based risk estimates might be barred from use in legal proceedings. Using a fictional scenario, I seek to show how group data have an obvious application to individual decisions. I also explain how misunderstanding the aims of risk assessment has led to mistakes about how, when, and why group data apply to individual instances. Although actuarially based statements about individuals' risk have many pitfalls, confidence intervals pose no barrier to using actuarial tools derived from group data to improve decision-making about individual instances.