Assessing the Calibration of Actuarial Risk Scales

Abstract

Assessing the predictive accuracy of actuarial risk assessment tools requires consideration of discrimination (the differences between recidivists and nonrecidivists) and calibration (the credibility of the recidivism rates associated with test scores or categories). Currently, there are no conventions for reporting calibration effect sizes for offender risk tools. This article explains one promising calibration effect size statistic (the Expected/Observed [E/O] index) and provides an illustrative example of how it can be calculated and interpreted. Briefly, the E/O index is the ratio of the expected number of recidivists to the observed number of recidivists. Guidance is provided for calculating the E/O index with fixed follow-up data as well as from survival data. This article also discusses alternative approaches to examining calibration and provides references to other studies using the E/O index to assess the calibration of offender risk scales.