Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses

Abstract

Objectives To evaluate meta-analyses with trial sequential analysis (TSA). TSA adjusts for random error risk and provides the required number of participants (information size) in a meta-analysis. Meta-analyses not reaching information size are analyzed with trial sequential monitoring boundaries analogous to interim monitoring boundaries in a single trial. Study Design and Setting We applied TSA on meta-analyses performed in Cochrane Neonatal reviews. We calculated information sizes and monitoring boundaries with three different anticipated intervention effects of 30% relative risk reduction (TSA 30%), 15% (TSA 15%), or a risk reduction suggested by low-bias risk trials of the meta-analysis corrected for heterogeneity (TSA LBHIS). Results A total of 174 meta-analyses were eligible; 79 out of 174 (45%) meta-analyses were statistically significant ( P < 0.05). In the significant meta-analyses, TSA 30% showed firm evidence in 61%. TSA 15% and TSA LBHIS found firm evidence in 33% and 73%, respectively. The remaining significant meta-analyses had potentially spurious evidence of effect. In the 95 statistically nonsignificant ( P ≥ 0.05) meta-analyses, TSA 30% showed absence of evidence in 80% (insufficient information size). TSA 15% and TSA LBHIS found that 95% and 91% had absence of evidence. The remaining nonsignificant meta-analyses had evidence of lack of effect. Conclusion TSA reveals insufficient information size and potentially false positive results in many meta-analyses.