Response to Lucas, Oishi, and Diener

Abstract

In their comment on our paper, Diener, Lucas, and Oishi (DLO) acknowledge the importance of our research. Yet, their extensive effort to provide a context for interpreting the findings—in particular, that our work contradicts a mass of earlier work—is incorrect, and their discussion entirely overlooks the paper’s new contribution. On the crucial issue of the size of the context effect, DLO argue that there is a large literature in psychology which consistently finds that context effects are small. All of the literature they present used test-retest correlations (reliability) to document the impact of context effects on wellbeing measures so that, when they say an effect is small, they mean that there was a high correlation between two sets of answers with and without the context item. As we explain in the penultimate paragraph of our paper, which on rereading seems crystal clear, such correlations are completely irrelevant to our result. (We therefore do not comment on their particular take on several seminal studies, but note that the matter is hardly as settled as they suggest.) Our results show that using wellbeing measures for tracking over time—in the same way that governments track GDP or unemployment—a context effect can introduce changes in responses that are large enough to completely undermine the validity and usefulness of the exercise. As we explain in that paragraph, such effects can occur with perfect reliability in DLO’s sense, for example if everyone changes their response by the same amount. DLO’s lengthy discussion simply misses the point.