Meta-analyses Of Binary Outcomes Research Articles

Synthesizing evidence from the earliest studies to support decision-making: To what extent could the evidence be reliable?

In evidence-based practice, new topics generally only have a few studies available for synthesis. As a result, the evidence of such meta-analyses raised substantial concerns. We investigated the robustness of the evidence from these earliest studies. Real-world data from the Cochrane Database of Systematic Reviews (CDSR) were collected. We emulated meta-analyses with the earliest 1 to 10 studies through cumulative meta-analysis from eligible meta-analyses. The magnitude and the direction of meta-analyses with the earliest few studies were compared to the full meta-analyses. From the CDSR, we identified 20,227 meta-analyses of binary outcomes and 7683 meta-analyses of continuous outcomes. Under the tolerable difference of 20% on the magnitude of the effects, the convergence proportion ranged from 24.24% (earliest 1 study) to 77.45% (earliest 10 studies) for meta-analyses of few earliest studies with binary outcomes. For meta-analyses of continuous outcomes, the convergence proportion ranged from 13.86% to 56.52%. In terms of the direction of the effects, even when only three studies were available at the earliest stage, the majority had the same direction as full meta-analyses; Only 19% for binary outcomes and 12% for continuous outcomes changed the direction as further evidence accumulated. Synthesizing evidence from the earliest studies is feasible to support urgent decision-making, and in most cases, the decisions would be reasonable. Considering the potential uncertainties, it is essential to evaluate the confidence of the evidence of these meta-analyses and update the evidence when necessary.

Rapid evidence synthesis approach for limits on the search date: How rapid could it be?

Rapid reviews have been widely employed to support timely decision-making, and limiting the search date is the most popular approach in published rapid reviews. We assessed the accuracy and workload of search date limits on the meta-analytical results to determine the best rapid strategy. The meta-analyses data were collected from the Cochrane Database of Systematic Reviews (CDSR). We emulated the rapid reviews by limiting the search date of the original CDSR to the recent 40, 35, 30, 25, 20, 15, 10, 7, 5, and 3 years, and their results were compared to the full meta-analyses. A random sample of 10% was drawn to repeat the literature search by the same timeframe limits to measure the relative workload reduction (RWR). The relationship between accuracy and RWR was established. We identified 21,363 meta-analyses of binary outcomes and 7683 meta-analyses of continuous outcomes from 2693 CDSRs. Our results suggested that under a maximum tolerance of 5% and 10% on the bias of magnitude, a limit on the recent 20 years can achieve good accuracy and at the same time save the most workload. Under the tolerance of 15% and 20% on the bias, a limit on the recent 10 years and 15 years could be considered. Limiting the search date is a valid rapid method to produce credible evidence for timely decisions. When conducting rapid reviews, researchers should consider both the accuracy and workload to make an appropriate decision.

Meta-analysis of binary outcomes via generalized linear mixed models: a simulation study

BackgroundSystematic reviews and meta-analyses of binary outcomes are widespread in all areas of application. The odds ratio, in particular, is by far the most popular effect measure. However, the standard meta-analysis of odds ratios using a random-effects model has a number of potential problems. An attractive alternative approach for the meta-analysis of binary outcomes uses a class of generalized linear mixed models (GLMMs). GLMMs are believed to overcome the problems of the standard random-effects model because they use a correct binomial-normal likelihood. However, this belief is based on theoretical considerations, and no sufficient simulations have assessed the performance of GLMMs in meta-analysis. This gap may be due to the computational complexity of these models and the resulting considerable time requirements.MethodsThe present study is the first to provide extensive simulations on the performance of four GLMM methods (models with fixed and random study effects and two conditional methods) for meta-analysis of odds ratios in comparison to the standard random effects model.ResultsIn our simulations, the hypergeometric-normal model provided less biased estimation of the heterogeneity variance than the standard random-effects meta-analysis using the restricted maximum likelihood (REML) estimation when the data were sparse, but the REML method performed similarly for the point estimation of the odds ratio, and better for the interval estimation.ConclusionsIt is difficult to recommend the use of GLMMs in the practice of meta-analysis. The problem of finding uniformly good methods of the meta-analysis for binary outcomes is still open.

Empirical Comparison of Publication Bias Tests in Meta-Analysis.

Decision makers rely on meta-analytic estimates to trade off benefits and harms. Publication bias impairs the validity and generalizability of such estimates. The performance of various statistical tests for publication bias has been largely compared using simulation studies and has not been systematically evaluated in empirical data. This study compares seven commonly used publication bias tests (i.e., Begg's rank test, trim-and-fill, Egger's, Tang's, Macaskill's, Deeks', and Peters' regression tests) based on 28,655 meta-analyses available in the Cochrane Library. Egger's regression test detected publication bias more frequently than other tests (15.7% in meta-analyses of binary outcomes and 13.5% in meta-analyses of non-binary outcomes). The proportion of statistically significant publication bias tests was greater for larger meta-analyses, especially for Begg's rank test and the trim-and-fill method. The agreement among Tang's, Macaskill's, Deeks', and Peters' regression tests for binary outcomes was moderately strong (most κ's were around 0.6). Tang's and Deeks' tests had fairly similar performance (κ > 0.9). The agreement among Begg's rank test, the trim-and-fill method, and Egger's regression test was weak or moderate (κ < 0.5). Given the relatively low agreement between many publication bias tests, meta-analysts should not rely on a single test and may apply multiple tests with various assumptions. Non-statistical approaches to evaluating publication bias (e.g., searching clinical trials registries, records of drug approving agencies, and scientific conference proceedings) remain essential.

Lack of Evidence for Dual Antiplatelet Therapy after Endovascular Arterial Procedures: A Meta-analysis

Dual antiplatelet therapy (DAPT) has mainly replaced mono antiplatelet therapy (MAPT) and is recommended after arterial endovascular revascularization. The aim of this meta-analysis was to summarize the available evidence for DAPT after endovascular revascularization throughout the arterial system. A systematic search was performed in Medline, Embase, and the Cochrane Register. Two reviewers independently performed data extraction and quality assessment using the Cochrane Collaboration risk of bias assessment tool. Included in the search were randomized controlled trials (RCTs) comparing DAPT with MAPT after endovascular procedures for the treatment of coronary, carotid, or peripheral artery disease, reporting at least one clinical outcome. Articles were excluded if patients received anticoagulation in addition to antiplatelet therapy in the post-procedural phase. The primary outcome was restenosis or stent thrombosis, and secondary outcomes were major adverse cardiac events (MACE), target lesion revascularization, cerebrovascular accident or transient ischemic attack, bleeding, and death. Meta-analyses of binary outcomes were performed using the random effects model and described as risk ratios (RRs) and 95% confidence intervals (95% CIs). Chi-square tests were used to test for heterogeneity. Nine articles were included in this study, involving lower limb peripheral arteries (1), carotid arteries (2), and coronary arteries (6). The pooled results of coronary trials showed a RR for restenosis with DAPT of 0.60 (95% CI 0.28-1.31) and for myocardial infarction 0.49 (95% CI 0.12-2.03). In the carotid artery trials the RR for restenosis was 0.22 (95% CI 0.04-1.20) and for peripheral arteries 1.02 (95% CI 0.56-1.82). A meta-analysis of bleeding risk of all the included trials showed a RR of 1.06 (95% CI 0.32-3.52) with DAPT. The available evidence comparing DAPT with MAPT after endovascular arterial revascularization is limited and the majority of trials were conducted in the cardiology field. No significant evidence for superiority of DAPT compared with MAPT was found, but there was also no evidence of an increased bleeding risk with DAPT over MAPT.

High statistical heterogeneity is more frequent in meta-analysis of continuous than binary outcomes

ObjectivesWe compared the distribution of heterogeneity in meta-analyses of binary and continuous outcomes. Study Design and SettingWe searched citations in MEDLINE and Cochrane databases for meta-analyses of randomized trials published in 2012 that reported a measure of heterogeneity of either binary or continuous outcomes. Two reviewers independently performed eligibility screening and data abstraction. We evaluated the distribution of I2 in meta-analyses of binary and continuous outcomes and explored hypotheses explaining the difference in distributions. ResultsAfter full-text screening, we selected 671 meta-analyses evaluating 557 binary and 352 continuous outcomes. Heterogeneity as assessed by I2 proved higher in continuous than in binary outcomes: the proportion of continuous and binary outcomes reporting an I2 of 0% was 34% vs. 52%, respectively, and reporting an I2 of 60–100% was 39% vs. 14%. In continuous but not binary outcomes, I2 increased with larger number of studies included in a meta-analysis. Increased precision and sample size do not explain the larger I2 found in meta-analyses of continuous outcomes with a larger number of studies. ConclusionsMeta-analyses evaluating continuous outcomes showed substantially higher I2 than meta-analyses of binary outcomes. Results suggest differing standards for interpreting I2 in continuous vs. binary outcomes may be appropriate.

The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews

BackgroundMost meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results.Methods and FindingsFor 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range −1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease.ConclusionsWhen at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.

An informed reference prior for between‐study heterogeneity in meta‐analyses of binary outcomes

It is well known that when a Bayesian meta-analysis includes a small number of studies, inference can be sensitive to the choice of prior for the between-study variance. Choosing a vague prior does not solve the problem, as inferences can be substantially different depending on the degree of vagueness. Moreover, because the data provide little information on between-study heterogeneity, posterior inferences for the between-study variance based on vague priors will tend to be unrealistic. It is thus preferable to adopt a reasonable, informed prior for the between-study variance. However, relatively little is known about what constitutes a realistic distribution. On the basis of data from the Cochrane Database of Systematic Reviews, this paper describes the distribution of between-study variance in published meta-analyses, and proposes some realistic, informed priors for use in meta-analyses of binary outcomes. It is hoped that these priors will improve the calibration of inferences from Bayesian meta-analyses.

Weighting bias in meta-analysis of binary outcomes

This article demonstrates that the weighting-according-to-the-variance method may introduce biases in meta-analyses of binary outcomes. The weighting favors studies that have certain frequencies of outcome events and weights given to studies of the same size may differ tens to thousands of times merely because of variations in the frequency. It also applies different standards to different measures of effect. Thus, the weighting may distort the combined result or even lead to contradictory conclusions when different measures of effect are used. Generally, the bias is more likely to arise when the effect is heterogeneous across the combined trials, the trials are conducted in populations of highly varied risks, the relative risk is used as the effect measure, the effect to be combined is small, any of the trials falls beyond the risk range of 20% and 80%, and/or the number of trials is small. Suggestions for detection and control of the bias are also given.