The psychometric paradox of practice effects due to retesting: Measurement invariance and stable ability estimates in the face of observed score changes

Abstract

The simple practice effects commonly observed when retaking general cognitive ability tests present a potential paradox. If observed score changes reflect real changes in g, we must revisit our understanding of its stability. Conversely, if observed score changes reflect something other than a true change in the underlying latent construct, this suggests that the assumption of measurement invariance (i.e., that the relation between the constructs and the indicators on the test remains stable) is unwarranted, calling into question many of the useful inferences based on ability test scores. However, using a construct level approach, the current study shows evidence for complete measurement invariance across repeated administrations, including equivalence of ability factor means and variances. These results indicate that the abilities reflected by g and narrow group factors, and the tests' assessment of those sources of variance, remain unchanged.