Meta-Analysis of the Effects of Computerized Adaptive Testing on the Motivation and Emotion of Examinees

Abstract

Abstract: Findings on the motivational and emotional effects of computerized adaptive testing (CAT) compared to nonadaptive testing (NCAT) have been heterogeneous and inconclusive so far. This meta-analysis aims to determine the specific effects of CAT better by analyzing several moderator variables related to the test, the examinee, and the test setting. To be included in the meta-analysis, studies needed to be original, correlational, quasi-experimental, or experimental studies in English, German, or Chinese, and they needed to compare CAT to NCAT regarding motivational and/or emotional states. A three-level random-effects model with motivation, negative emotion, and positive emotion as dependent variables was estimated. The risk of bias and publication bias were examined. Twenty-seven studies with a total of 190 effect sizes were analyzed. The overall effect of CAT was not significant for motivation and negative emotion and significant for positive emotion after publication bias correction. The moderator analysis showed significant moderator effects for “test speededness” on motivation and positive emotion, “instantaneous feedback on given responses” and “possibility of response revision” on positive emotion, and “average difficulty of presented items” on negative emotion. For CAT configurations, it is recommended to avoid speededness, present easy items, allow for response revision, and provide feedback.