Effect Sizes and their Intervals: The Two-Level Repeated Measures Case

Abstract

Probability coverage for eight different confidence intervals (CIs) of measures of effect size (ES) in a two-level repeated measures design was investigated. The CIs and measures of ES differed with regard to whether they used least squares or robust estimates of central tendency and variability, whether the end critical points of the interval were obtained using a theoretical or an empirical sampling distribution, and whether the ESs used a pooled or nonpooled estimate of error variability. These intervals were compared when data were obtained from both normal and nonnormal distributions and when the population magnitude of effect, size of sample, and variance heterogeneity were varied. Itwas found that the ESs and intervals that used robust estimators and critical values were obtained through a bootstrap method better at controlling the probability coverage (i.e., within [.925, .975]).