Publication Bias in Special Education Meta-Analyses

Abstract

Publication bias involves the disproportionate representation of studies with large and significant effects in the published research. Among other problems, publication bias results in inflated omnibus effect sizes in meta-analyses, giving the impression that interventions have stronger effects than they actually do. Although evidence suggests that publication bias exists in other fields, research has not examined the issue in special education. In this study, we examined the inclusion of gray literature, testing for publication bias, the extent to which publication bias exists, the relation of including gray literature to the presence of publication bias, and differences in effect size magnitude for gray literature and published studies among 109 meta-analyses published in special education journals. We found the following: (a) 42% of meta-analyses included gray literature, (b) 33% examined publication bias, (c) meta-analyses not including gray literature were more likely to reflect publication bias, and (d) published studies had larger effect sizes than gray literature. We discuss implications and recommendations for research and practice.