The effect of question order on evaluations of test performance: Can the bias dissolve?

Abstract

Question difficulty order has been shown to affect students' global postdictions of test performance. We attempted to eliminate the bias by letting participants experience the question order manipulation multiple times. In all three experiments, participants answered general knowledge questions and self-evaluated their performance. In Experiment 1, participants studied questions and answers in easy-hard or hard-easy question order prior to taking a test in the same order. In Experiment 2, participants took the same test twice in the opposite question order (easy-hard then hard-easy, or hard-easy then easy-hard). In Experiment 3, participants took two different tests in the opposite question order (easy-hard then hard-easy, or hard-easy then easy-hard). In all three experiments, we were unable to eliminate the bias, which suggests that repeated exposure is insufficient to overcome a strong initial anchor.