Optimizing Detection of True Within-Person Effects for Intensive Measurement Designs: A Comparison of Multilevel SEM and Unit-Weighted Scale Scores.

Abstract

Intensive repeated measurement designs are frequently used to investigate within-person variation over relatively brief intervals of time. The majority of research utilizing these designs relies on unit-weighted scale scores, which assume that the constructs are measured without error. An alternative approach makes use of multilevel structural equation models (MSEM), which permit the specification of latent variables at both within-person and between-person levels. These models disaggregate measurement error from systematic variance, which should result in less biased within-person estimates and larger effect sizes. Differences in power, precision, and bias between multilevel unit-weighted modelsand MSEMs were compared through a series of Monte Carlo simulations. Results based on simulated data revealed that precision was consistently poorer in the MSEMs than the unit-weighted models, particularly when reliability was low. However, the degree of bias was considerably greater in the unit-weighted model than the latent variable model. Although the unit-weighted model consistently underestimated the effect of a covariate, it generally had similar power relative to the MSEM model due to the greater precision. Considerations for scale development and the impact of within-person reliability are highlighted.