Belief bias in the perception of sample size adequacy

Abstract

In two experiments participants were instructed to set aside their own, complete knowledge of a statistical population parameter and to take the perspective of an agent whose knowledge was limited to a random sample. Participants rated the appropriateness of the agent's conclusion about the adequacy of the sample size (which, objectively, was more than adequate). They also rated the agent's intelligence. Whereas previous work suggests that unbelievable statistical conclusions impact reasoning by provoking critical thought which enhances the detection of research flaws, the present studies presented participants an unflawed scenario designed to assess effects of believability on bias. The results included the finding that participants’ complete knowledge did indeed bias their perceptions not only of the adequacy of the sample size, but also of the rationality of the agent drawing the conclusion from the sample. The findings were interpreted in the context of research on belief bias, social attribution, and Theory of Mind.