Graphical Augmentations to the Funnel Plot to Assess the Impact of a New Study on an Existing Meta-Analysis

Abstract

Funnel plots are currently advocated to investigate the presence of publication bias (and other possible sources of bias) in meta-analysis. A previously described augmentation to the funnel plot—to aid its interpretation in assessing publication biases—is the addition of statistical contours indicating regions where studies would have to be for a given level of significance, as implemented in the Stata package confunnel by Palmer et al. (2008, Stata Journal 8: 242–254). In this article, we describe the implementation of a new range of overlay augmentations to the funnel plot, many described in detail recently by Langan et al. (2012, Journal of Clinical Epidemiology 65: 511–519). The purpose of these overlays is to display the potential impact a new study would have on an existing meta-analysis, providing an indication of the robustness of the meta-analysis to the addition of new evidence. Thus these overlays extend the use of the funnel plot beyond assessments of publication biases. Two main graphical displays are described: 1) statistical significance contours, which define regions of the funnel plot where a new study would have to be located to change the statistical significance of the meta-analysis; and 2) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. We present the extfunnel command, which implements the methods of Langan et al. (2012, Journal of Clinical Epidemiology 65: 511–519), and, furthermore, we extend the graphical displays to illustrate the impact a new study has on lower and upper confidence interval values and the confidence interval width of the pooled meta-analytic result. We also describe overlays for the impact of a future study on user-defined limits of clinical equivalence. We implement inverse-variance weighted methods by using both explicit formulas for contour lines and a simulation approach optimized in Mata.