Abstract
Effect sizes are an important outcome of quantitative research, but few guidelines exist that explain how researchers can determine which effect sizes are meaningful. Psychologists often want to study effects that are large enough to make a difference to people's subjective experience. Thus, subjective experience is one way to gauge the meaningfulness of an effect. We propose and illustrate one method for how to quantify the smallest subjectively experienced difference—the smallest change in an outcome measure that individuals consider to be meaningful enough in their subjective experience such that they are willing to rate themselves as feeling different—using an anchor-based method with a global rating of change question applied to the positive and negative affect scale. We provide a step-by-step guide for the questions that researchers need to consider in deciding whether and how to use the anchor-based method, and we make explicit the assumptions of the method that future research can examine. For researchers interested in people's subjective experiences, this anchor-based method provides one way to specify a smallest effect size of interest, which allows researchers to interpret observed results in terms of their theoretical and practical significance.
Highlights
Effect sizes are an important outcome of quantitative research, but few guidelines exist that explain how researchers can determine which effect sizes are meaningful
Cohen's dav can in theory be more compared across within- and between-subjects designs, future research should examine whether subjectively experienced differences in different outcome measures can be assumed to be constant across within and between designs
We provided an illustrative example by estimating the minimum subjectively experienced difference for positive and negative affect as measured by the Positive and Negative Affect Scale (PANAS)
Summary
Effect sizes are an important outcome of quantitative research, but few guidelines exist that explain how researchers can determine which effect sizes are meaningful. For researchers interested in people’s subjective experiences, this anchor-based method provides one way to specify a smallest effect size of interest, which allows researchers to interpret observed results in terms of their theoretical and practical significance. How Determining a SESOI Improves Research By examining whether an observed effect size is not just statistically significant, but larger than the smallest effect size of interest (SESOI), researchers can draw conclusions about whether the observed effect is theoretically or practically significant This can help to prevent the common misinterpretation of ‘statistically significant’ as ‘meaningful’, which is becoming increasingly important given the rise of big data and the uptake of large-scale collaborative projects (e.g., Klein et al, 2014, 2018; Moshontz et al, 2018), where trivially small differences can be statistically significant. To be able to reap those benefits, researchers need methods to determine their SESOI
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
