Do Published Studies Yield Larger Effect Sizes than Unpublished Studies in Education and Special Education? A Meta-review

Abstract

Meta-analyses are used to make educational decisions in policy and practice. Publication bias refers to the extent to which published literature is more likely to have statistically significant results and larger sample sizes than studies that do not make it through the publication process. The purpose of the present study is to estimate the extent to which publication bias is present in a broad set of education and special education journals. We reviewed 222 meta-analyses to describe the prevalence of publication bias tests, and further identified 29 that met inclusion criteria for effect size extraction. Descriptive data reveal that 58% of meta-analyses (n = 128) documented no effort to test for possible publication bias, and analyses of 72 difference statistics revealed that published studies were associated with significantly larger effect sizes than unpublished studies (d = 0.64). Exploratory moderator analyses revealed that effect size metric was a significant predictor of the difference between published and unpublished studies.