Trial Number and Sample Size Do Not Affect the Accuracy of the I2-Point Estimate for Testing Selection Bias Risk in Meta-Analyses.

Abstract

Aim This study aims to establish the test sensitivity and specificity of the I2-point estimate for testing selection bias in meta-analyses under the condition of large versus small trial sample size and large versus small trial number in meta-analyses and to test the null hypotheses that the differences are not statistically significant. Material and methods Simulation trials were generated in MS Excel (Microsoft Corp., Redmond, WA), each consisting of a sequence of subject ID (accession) numbers representing trial subjects, a random sequence of allocation to group A or B, and a random sequence of a simulated baseline variable ("age") per subject, ranging from 50 to 55. These simulation trials were included infive types of meta-analyses with large/small numbers of trials, as well as trials with large and small sample sizes. Half of the meta-analyses were artificially biased. All meta-analyses were tested using the I2-point estimate. The numbers of true positive (TP), false positive (FP), false negative (FN), and true negative (TN) test results were established. From these, the test sensitivity and specificity were computed for each of the meta-analysis types and compared. Results All non-biased meta-analyses yielded true negative, and all biased meta-analyses yielded true positive test results, regardless of trial number and trial sample size. No false positive or false negative test results were observed. Accordingly, test sensitivities and specificities of 100% for all meta-analysis types were established, and thus, both null hypotheses failed to be rejected. Conclusion The results suggest that trial number and sample size in a baseline variable meta-analysis do not affect the test accuracy of the I2-point estimate.