Validity of the Anchor in Estimating the Smallest Subjectively Experienced Difference: Presenting an Anchor-Item Before vs After the Outcome Measure

Abstract

In some fields of research, psychologists are interested in effect sizes that are large enough to make a difference to people’s subjective experience. Recently, an anchor-based method using a global rating of change was proposed as a way to quantify the smallest subjectively experienced difference—the smallest numerical difference in the outcome measure that, on average, corresponds to reported changes in people’s subjective experience. According to the method, the construct of interest is measured on two occasions (Time 1 and Time 2). At Time 2, people also use an anchor-item to report how much they experienced a change in the construct. Participants are then categorized as those who stayed the same, those who changed a lot, and those who changed a little. The average change score for those who changed a little is the estimate of the smallest subjectively experienced difference. In the present study, I examined two aspects of the method’s validity. First, I tested whether presenting the anchor-item before or after the Time 2 outcome measure influences the results. The results suggest that any potential influence of the anchor-position, assuming there is an influence, is likely to be small. Second, I examined the anchor-item’s validity correlations when the delay between Time 1 and 2 is one day to also see if the pattern is similar to past research where the delay was two and five days. The observed pattern of validity correlations was very similar. I note directions for future research.