The Hard–Easy Effect in Subjective Probability Calibration

Abstract

Listen

Translate article

Four different factors have been used to explain the hard–easy effect in subjective probability judgments, in which calibration goes systematically from over- to underconfidence as task difficulty decreases. The factors are biased judgment, lack of suitable adjustment of response criteria (decision variable partition model), biased choice of stimulus material (ecological model), and random error in judgment (error model). Although there is strong evidence for judgmental bias, it seems unlikely to account fully for the systematic character of the effect. We describe the judgment models that emphasize the other three factors and then present two new experiments and a further analysis of the calibration problem. The findings of these experiments and the analysis support the response criteria approach and indicate that the ecological model, and biased choice of stimulus material, is not correct. Since error by itself cannot fully explain observed underconfidence phenomena, we conclude that the most likely explanation of the hard–easy effect is in terms of response criteria. A simulation of the decision variable partition model shows that its behavior closely matches the observed relation between overconfidence and proportion correct for a large sample of experimental results.