The Effect of Base Rate on Calibration of Subjective Probability for True-False Questions: Model and Experiment

Abstract

Abstract The degree of calibration of subjective probabilities of events is the extent to which the observed proportion of events that occur agrees with the assigned probability values. The decision variable partition model of calibration is reviewed tutorially. It shows how numerical subjective probabilities for discrete events can be related to the perceived truth of propositions. It has been able to explain a number of experimental findings about calibration of subjective probabilities of correct response to two-alternative multiple-choice questions and to questions to which the respondent supplies the answer. In this paper, the model's predictions for true-false items axe derived, and an experiment testing them is reported. The model predicts that when the subjective probability that items are true is assessed, there will be a specific effect of the base rate, the proportion of true items, but that there will be no effect when the respondent decides true or false and then reports a subjective probability of being correct. A systematic effect of task difficulty is predicted in both cases. The experimental results are in close agreement with the model's predictions.