A Testing Load: Investigating Test Mode Effects on Test Score, Cognitive Load and Scratch Paper Use with Secondary School Students

Abstract

The aim of the present study is to reconcile previous findings (a) that testing mode has no effect on test outcomes or cognitive load (Comput Hum Behav 77:1–10, 2017) and (b) that younger learners’ working memory processes are more sensitive to computer-based test formats (J Psychoeduc Assess 37(3):382–394, 2019). We addressed key methodological limitations in past cognitive load research by employing a repeated measures design with 263, year 9 (aged 13–14) science students in Western Australia. Question difficulty (intrinsic cognitive load) and test mode (extraneous cognitive load) were manipulated to measure changes in test performance, cognitive load and scratch paper use on equivalent paper and computer-based versions of an Ohm’s Law revision quiz. Hierarchical linear modelling indicated significantly higher paper-based test performance on difficult questions in addition to greater cognitive load and scratch paper use for all paper questions. Testing mode effects on test score, as well as both measures of cognitive load, were not significant when controlling for working memory capacity, although the testing mode*question difficulty interaction remained significant. Together, these results contradict previous findings that computer-based testing can be implemented without consequence for all learners. With the increased use of computer-based testing in national and international-level assessments, these findings warrant further research into the effect of different testing modes on school-aged students.