Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy.

Abstract

Given a data set about an individual or a group (e.g., interviewer ratings, life history or demographic facts, test results, self-descriptions), there are two modes of data combination for a predictive or diagnostic purpose. The method relies on human judgment that is based on informal contemplation and, sometimes, discussion with others (e.g., case conferences). The mechanical method involves a formal, algorithmic, objective procedure (e.g., equation) to reach the decision. Empirical comparisons of the accuracy of the two methods (136 studies over a wide range of predictands) show that the mechanical method is almost invariably equal to or superior to the method: Common antiactuarial arguments are rebutted, possible causes of widespread resistance to the comparative research are offered, and policy implications of the statistical method's superiority are discussed. In 1928, the Illinois State Board of Parole published a study by sociologist Burgess of the parole outcome for 3,000 criminal offenders, an exhaustive sample of parolees in a period of years preceding. (In Meehl, 1954/1996, this number is erroneously reported as 1,000, a slip probably arising from the fact that 1,000 cases came from each of three Illinois prisons.) Burgess combined 21 objective factors (e.g., nature of crime, nature of sentence, chronological age, number of previous offenses) in unweighted fashion by simply counting for each case the number of factors present that expert opinion considered favorable or unfavorable to successful parole outcome. Given such a large sample, the predetermination of a list of relevant factors (rather than elimination and selection of factors), and the absence of any attempt at optimizing weights, the usual problem of crossvalidation shrinkage is of negligible importance. Subjective, impressionistic, clinical judgments were also made by three prison psychiatrists about probable parole success. The psychiatrists were slightly more accurate than the actuarial tally of favorable factors in predicting parole success, but they were markedly inferior in predicting failure. Furthermore, the actuarial tally made predictions for every case, whereas the psychiatrists left a sizable fraction of cases undecided. The conclusion was clear that even a crude actuarial method such as this was superior to judgment in accuracy of prediction. Of course, we do not know how many of the 21 factors the psychiatrists took into account, but all were available to