Explanations of Statistical Concepts: Can they Penetrate the Haze of Bayes?

Abstract

SummaryThis experiment tested the ability of logical explanations to promote the use of statistical concepts. Sixty physicians in four experimental groups, matched for level of medical statistical training, were given a questionnaire that assessed their ability to rank the diagnosticity of various diagnostic test results. Each question was designed to test physician understanding of the impact of a particular statistical parameter, e.g., sensitivity and specificity, test-retest reliability. Logical explanations of the sources of test error increased the percentage of answers judged to be correct on the basis of published data. This was confirmed for a variety of questions. They tended to be less useful, however, when physicians expressed misinterpretations of statistical concepts or test patterns were not correctly perceived. The implications of these findings for medical education and for the design of computer-based explanation systems are clarified.