Online test administration results in students selecting more responses to multiple-choice-multiple-response items

Abstract

We developed the Physics Inventory of Quantitative Literacy (PIQL) to assess students’ quantitative reasoning in introductory physics contexts. The PIQL includes several “multiple-choice-multiple-response” (MCMR) items (i.e., multiple-choice questions for which more than one response may be selected) as well as traditional single-response multiple-choice items. In this paper, we discuss differences in performance on MCMR items that seems to result from differences in administration method (paper versus online). In particular, we find a tendency for “clickiness” in online administration: students choose more responses to MCMR items when taking the electronic version of the assessment. Student performance on single-response multiple-choice items was not affected by administration method. These results suggest that MCMR items may provide a unique opportunity to probe differences in online and on-paper administration of low-stakes assessments.Received 22 August 2022Accepted 20 December 2022DOI:https://doi.org/10.1103/PhysRevPhysEducRes.19.013101Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasAssessmentInstructional strategiesTechnologyPhysics Education Research