Influences of Carry-Over Effects Across Scales on Mediation Analyses

Abstract

ABSTRACT Many education tests and psychological surveys elicit respondent views of similar constructs across scenarios (e.g., story followed by multiple choice questions) by repeating common statements across scales (one-statement-multiple-scale, OSMS). However, a respondent’s earlier responses to the common statement can affect later responses to it (recursive carry-over effect), which violates the local item independence assumption of most measurement models. Ignoring this carry-over effect can bias both estimated item parameters for later scales and relations among scales. This study investigates the consequences of model misspecification for OSMS questions in mediation analyses via real data from the Teaching and Learning International Survey (TALIS) and simulations. The results showed that failing to consider carry-over effects in a mediation model yielded biased relations between latent variables. Moreover, our MMCEO mediation model corrected the inflated correlations caused by carry-over effects and yielded superior estimates of the mediation effects. We discuss implications for appropriately addressing carry‑over effects.