A review of the reliability and validity of OPCRIT in relation to its use for the routine clinical assessment of mental health patients

Abstract

The OPCRIT program is a symptom checklist with accompanying algorithms producing operationally defined diagnoses. We undertook a review of studies which had used OPCRIT and had reported statistics concerning its reliability and validity, producing summary measures from 44 studies. The first main measure of interest was inter-rater reliability where mean kappa values indicated that agreement between raters was "substantial" with a marginal improvement at the diagnostic (0.76) versus individual item (0.69) level. The second main measure of interest was convergent validity - the agreement between OPCRIT and clinical diagnoses. Most studies reported these figures as concordance rates suggesting mean agreement, unadjusted for chance, of 69%. Very few studies used the chance-adjusted kappa statistic but where this was used agreement was "fair" (0.39). Agreement between OPCRIT and other research diagnoses was "moderate" (0.60). We also considered differences between the way OPCRIT has traditionally been used in research settings and the naturalistic manner in which it will be employed in the hospital ward. This review provides a summary of the reliability and validity of OPCRIT, which will be considered during the preparation for its use in the routine characterization of mental health patients in clinical settings.