Reducing the dependence of clinical judgment on the immediate context: Effects of number of categories and type of anchors.

Abstract

Two experiments explored methods for standardizing ratings of the psychopathology of clinical case histories. In both experiments, the same case histories were rated as more pathological when mostly mild rather than severe cases were presented as the immediate context. Psychometric analyses demonstrated that this type of contextual effect is a potentially important source of unreliability in clinical judgment. In Experiment 1, increasing the number of points in the rating scale from 3 to either 7 or 100 significantly reduced the effects of the immediate context. Ratings were parsimoniously modeled by Parducci's (1983) range-frequency theory. In Experiment 2, providing verbal anchors in the form of either detailed DSM descriptions for each rating category or sample case histories for the two end-categories increased the reliability of the ratings by reducing the effects of the immediate contexts; however, these reductions occurred only when the ranges of the immediate contexts had been severely restricted. According to the range-frequency analysis, verbal anchors served to equate the endpoints of the subjective range for the different contextual conditions. Comparison with previous research suggests that the anchors also reduced the effects of the sequential position in which clinical cases appear. We therefore recommend that studies of the reliability of behavioral assessment techniques take into account the effects of differences in context.