Abstract
BackgroundThresholds for statistical significance when assessing meta-analysis results are being insufficiently demonstrated by traditional 95% confidence intervals and P-values. Assessment of intervention effects in systematic reviews with meta-analysis deserves greater rigour.MethodsMethodologies for assessing statistical and clinical significance of intervention effects in systematic reviews were considered. Balancing simplicity and comprehensiveness, an operational procedure was developed, based mainly on The Cochrane Collaboration methodology and the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) guidelines.ResultsWe propose an eight-step procedure for better validation of meta-analytic results in systematic reviews (1) Obtain the 95% confidence intervals and the P-values from both fixed-effect and random-effects meta-analyses and report the most conservative results as the main results. (2) Explore the reasons behind substantial statistical heterogeneity using subgroup and sensitivity analyses (see step 6). (3) To take account of problems with multiplicity adjust the thresholds for significance according to the number of primary outcomes. (4) Calculate required information sizes (≈ the a priori required number of participants for a meta-analysis to be conclusive) for all outcomes and analyse each outcome with trial sequential analysis. Report whether the trial sequential monitoring boundaries for benefit, harm, or futility are crossed. (5) Calculate Bayes factors for all primary outcomes. (6) Use subgroup analyses and sensitivity analyses to assess the potential impact of bias on the review results. (7) Assess the risk of publication bias. (8) Assess the clinical significance of the statistically significant review results.ConclusionsIf followed, the proposed eight-step procedure will increase the validity of assessments of intervention effects in systematic reviews of randomised clinical trials.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2288-14-120) contains supplementary material, which is available to authorized users.
Highlights
Systematic reviews summarise the results from randomised clinical trials
If followed, the proposed eight-step procedure will increase the validity of assessments of intervention effects in systematic reviews of randomised clinical trials
We acknowledge that a systematic review with meta-analysis cannot be conducted with the same scientific cogency as a randomised clinical trial with pre-defined high-quality methodology addressing an a priori and quantitatively hypothesised intervention effect
Summary
Systematic reviews summarise the results from randomised clinical trials. Meta-analysis is the main statistical method used in systematic reviews to analyse pooled results of trials [1,2]. Some claim that results of systematic reviews should be considered hypothesis-generating and should primarily serve the purpose of designing future randomised clinical trials [3,4,5]. Others consider systematic reviews with meta-analysis the highest level of evidence assessing the effects of healthcare interventions [1,2]. We acknowledge that a systematic review with meta-analysis cannot be conducted with the same scientific cogency as a randomised clinical trial with pre-defined high-quality methodology addressing an a priori and quantitatively hypothesised intervention effect. Assessment of intervention effects in systematic reviews with meta-analysis deserves greater rigour
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
