Evaluating Response Shift in Statistical Mediation Analysis.

Abstract

Researchers and prevention scientists often develop interventions to target intermediate variables (known as mediators) that are thought to be related to an outcome. When researchers target a mediating construct measured by self-report, the meaning of self-report measure could change from pretest to posttest for the individuals who received the intervention - which is a phenomenon referred to as response shift. As a result, any observed changes on the mediator measure across groups or across time might reflect a combination of true change on the construct and response shift. Although previous studies have focused on identifying the source and type of response shift in measures after an intervention, there has been limited research on how using sum scores in the presence of response shift affects the estimation of mediated effects via statistical mediation analysis, which is critical for explaining how the intervention worked. In this paper, we focus on recalibration response shift, which is a change in internal standards of measurement, which affects how respondents interpret the response scale. We provide background on the theory of response shift and the methodology used to detect response shift (i.e., tests of measurement invariance). Additionally, we use simulated datasets to provide an illustration of how recalibration in the mediator can bias estimates of the mediated effect and also impact type I error and power.