Detecting Publication Bias in Meta-analyses: A Case Study of Fluctuating Asymmetry and Sexual Selection.

Abstract

Those familiar with human nature and the publication process acknowledge that biases due to selective reporting of results are likely widespread in all fields of academic inquiry that depend on tools of statistical inference (e.g., see Begg and Berlin 1988; Iyengar and Greenhouse 1988; and the extensive discussion following each). However, although qualitative and quantitative methods exist for assessing the prevalence of selective reporting, and selective reporting has been studied in the medical and social sciences (Begg 1994, and references therein), the issue has received little attention in recent meta-analyses of ecological and evolutionary patterns. Clearly, “if publication bias is present, and if it operates in the same direction for all studies (as is likely), then [meta-analysis] is likely not only to produce biased summary estimates but also to produce estimates which are apparently precise and accurate leading to conclusions which may not only be wrong but appear convincing” (Begg and Berlin 1988, p. 437). To biologists unacquainted with the formal study of publication patterns, the terms “selective reporting” (statistical significance of an outcome influences its likelihood of being reported or published) and “publication bias” (the inflation of average effect size due to selective reporting) may imply a conscious intent to deceive, but this