Rater experience and the predictive validity of Psychopathy Checklist: Youth Version scores

Abstract

We compared the predictive validity of Psychopathy Checklist: Youth Version (PCL:YV) scores assigned by a licensed clinician to scores assigned by a graduate student across a sample of 82 juvenile offenders. Although both raters completed in-depth training and practice scoring cases, the graduate student had no prior clinical experience. The raters showed a high level of agreement in their scoring for 11 reliability check cases (intraclass correlation coefficient, ICCA,1 = .90 for PCL:YV Total score), but the scores assigned by the licensed clinician were better predictors of post-release recidivism (area under the curve, AUC = .77) than those assigned by the graduate student (AUC = .45). There was more variability in the scores assigned by the licensed clinician than those assigned by the graduate student, suggesting that more experienced clinicians’ willingness to assign both high and low scores may help explain rater differences in predictive validity.