Validating light phenomena conceptual assessment through the lens of classical test theory and item response theory frameworks

Abstract

The light phenomena conceptual assessment (LPCA) is a conceptual survey of light phenomena that has been recently established by physics education research (PER) scholars. Studying the LPCA psychometric properties is imperative to inform its measurement validity to potential LPCA users as well as general educational researchers. Classical test theory (CTT) and item response theory (IRT) are two popular statistical frameworks that can be utilized to explore the LPCA measurement validity. To our knowledge, no PER studies have attempted to make a head-to-head comparison of these methods while validating the LPCA. This study is the first to delineate the LPCA measurement by statistically comparing CTT- and IRT-based analyses. The LPCA dataset was drawn from physics students from eight secondary schools presented by Ndihokubwayo and Uwamahoro (2020 Phys. Educ. 55 035009). Our results accomplish harmony between the CTT and IRT arguments to estimate the LPCA item performance and student ability probed by the LPCA. They support the idea that the LPCA may be used as an inventory for evaluating conceptual understanding of light phenomena from the low to high ability range of students, and even some LPCA items should be flagged based on CTT- and IRT-based validity arguments. Special considerations for further refinement related to the discriminating power of problematic LPCA items are discussed.