The meta-analysis: supportive or illuminating?

Abstract

Many of us use statistical methods like an unsteady person might use a lamppost—more for support (of our pre-conceived notions) than for illumination. After all, Yogi Bera once said, ‘If I didn't believe it, I wouldn't have seen it.’ So it is easy to scoff at some of the flimsy constructs our colleagues have brought forward to ‘illuminate’ important issues that have not been answered by definitive trial data. In no case has the derision reached as high a level as with the meta-analysis, an attempt to compile data from small trials to answer important clinical questions. Though there may be good scientific rationale for such an idea all of us enjoy pointing to numerous examples in which poorly executed meta-analyses led to conclusions that were swept away later by rigorous clinical trial results. The truth is that the meta-analysis, like any statistical method, is susceptible to error when performed incorrectly.1,2 And one of the most common mistakes is study selection. Inclusion of poor studies and exclusion of valuable ones can dramatically affect the quality of the analysis to the point of reaching conclusions that are exactly contrary to the truth. A second potential weakness of this methodology is that, even with the compilation of well-executed trials, numbers may still be insufficient to reach … *Corresponding author: Suite 558, Medical Office Building East, 100 Lancaster Avenue, Wynnewood, PA 19096, USA. Tel: +1 610 645 2682; fax: +1 610 896 0643. E-mail address : koweypr{at}mlhheart.org