Estimating the bias in meta analysis estimates for continuous data with non-random missing study variance

Abstract

When study variances are not reported or ‘missing”, it is common practice in meta analysis to assume that the missing variances are missing completely at random (MCAR). In practice, however, it is possible that the variances are not missing completely at random (NMAR). In this paper, we examine, analytically, the biases introduce in the meta analysis estimates when the missing study variances occur with non-random missing mechanism (MNAR), namely, when the magnitude of the missing variances are mostly larger than those that are reported. In meta analysis, this is more likely to occur in studies which carry larger variances. We looked at two common approaches in handling this problem, namely, the missing variances are imputed using the mean imputation, and the studies with missing study-variances are omitted from the analysis. The results suggest that for the estimate of the variance of the effect size, if the magnitude of the study-variances that are missing are mostly larger than those that are reported, the variance of the effect size will be underestimated. Thus under MNAR, the mean imputation gives false impression of precision as the estimated variance of the overall effect is too small. On the other hand, if the missing variances are mostly smaller, the variance will be overestimated.