Estimating Reliability of Within-Person Couplings in a Multilevel Framework

Abstract

Within-person couplings play a prominent role in psychological research and previous studies have shown that interindividual differences in within-person couplings predict future behavior. For example, stress reactivity—operationalized as the within-person coupling of stress and positive or negative affect—is an important predictor of various (mental) health outcomes and has often been assumed to be a more or less stable personality trait. However, issues of reliability of these couplings have been largely neglected so far. In this work, we present an estimate for the reliability of within-person couplings that can be easily obtained using the user-modifiable R code accompanying this work. Results of a simulation study show that this index performs well even in the context of unbalanced data due to missing values. We demonstrate the application of this index in a measurement burst study targeting the reliability and test–retest correlation of stress reactivity estimates operationalized as within-person couplings in a daily diary design. Reliability and test–retest correlations of stress reactivity estimates were rather low, challenging the implicit assumption of stress reactivity as a stable person-level variable. We highlight key factors that researchers planning studies targeting interindividual differences in within-person couplings should consider to maximize reliability.