A comparison of bootstrap approaches for homogeneity test based on dichotomous outcome in meta-analysis

Abstract

Meta-analysis is a useful research methodology which combines results of multiple studies to estimate means of outcomes of interest. To effectively conduct a meta-analysis, it is critical to assess whether study homogeneity is a reasonable assumption. In literature, many approaches have been proposed to estimate the between-study variance and test the assumption of study homogeneity. One approach is the parametric bootstrap method, which has the disadvantage of underestimating type I error rates at small sample size. In this article, we propose a novel bootstrap method to improve the shortcomings of the parametric bootstrap. We focus on testing homogeneity of the odds-ratio in different studies. In our approach, we first estimate response probabilities in the 2 2 tables under the assumption of homogeneity and then generate bootstrap samples based on the estimated probabilities. The performance of the novel bootstrap is evaluated through simulation studies conducted based on Cochran Q-statistics and several commonly used heterogeneity estimators. Simulation results show that Cochran Q-statistics based on both bootstrap methods properly control type I error rate except in the situation where both event probability and sample size are small. In general, the novel bootstrap tends to overestimate type I error while the parametric bootstrap tends to underestimate type I error. However, overestimation does not occur when using the novel bootstrap combined with tests based on the ML and REML. In this case, the novel bootstrap method controls type I error rates better than the parametric bootstrap and is as powerful as the parametric bootstrap. We illustrate the proposed method using two real data sets.