Is using multiple imputation better than complete case analysis for estimating a prevalence (risk) difference in randomized controlled trials when binary outcome observations are missing?

Mavuto Mukaka,Sarah A White,Victor Mwapasa,E Brian Faragher,Dianne J Terlouw,Linda Kalilani-Phiri

doi:10.1186/s13063-016-1473-3

Abstract

BackgroundMissing outcomes can seriously impair the ability to make correct inferences from randomized controlled trials (RCTs). Complete case (CC) analysis is commonly used, but it reduces sample size and is perceived to lead to reduced statistical efficiency of estimates while increasing the potential for bias. As multiple imputation (MI) methods preserve sample size, they are generally viewed as the preferred analytical approach.We examined this assumption, comparing the performance of CC and MI methods to determine risk difference (RD) estimates in the presence of missing binary outcomes. We conducted simulation studies of 5000 simulated data sets with 50 imputations of RCTs with one primary follow-up endpoint at different underlying levels of RD (3–25 %) and missing outcomes (5–30 %).ResultsFor missing at random (MAR) or missing completely at random (MCAR) outcomes, CC method estimates generally remained unbiased and achieved precision similar to or better than MI methods, and high statistical coverage. Missing not at random (MNAR) scenarios yielded invalid inferences with both methods. Effect size estimate bias was reduced in MI methods by always including group membership even if this was unrelated to missingness. Surprisingly, under MAR and MCAR conditions in the assessed scenarios, MI offered no statistical advantage over CC methods.ConclusionWhile MI must inherently accompany CC methods for intention-to-treat analyses, these findings endorse CC methods for per protocol risk difference analyses in these conditions. These findings provide an argument for the use of the CC approach to always complement MI analyses, with the usual caveat that the validity of the mechanism for missingness be thoroughly discussed. More importantly, researchers should strive to collect as much data as possible.

Highlights

Missing outcomes can seriously impair the ability to make correct inferences from randomized controlled trials (RCTs)
Missing outcome observations missing at random (MAR), missing completely at random (MCAR) and Missing not at random (MNAR): 60 % versus 85 % efficacy When the missing outcome setting was MAR, effect size estimates became increasingly inefficient as the proportion of missing outcome observations increased
This study has demonstrated that in the presence of missing binary outcome observations in an RCT with a single follow-up endpoint of interest, Complete case (CC) and multiple imputation (MI) analysis methods performed very under the three missingness assumptions examined, except when an inappropriate imputation model was adopted, in which case the MI risk difference (RD) estimates obtained were generally inferior to those generated by a CC analysis

Summary

Introduction

Missing outcomes can seriously impair the ability to make correct inferences from randomized controlled trials (RCTs). As multiple imputation (MI) methods preserve sample size, they are generally viewed as the preferred analytical approach We examined this assumption, comparing the performance of CC and MI methods to determine risk difference (RD) estimates in the presence of missing binary outcomes. A well-designed and conducted RCT provides an efficient and unbiased estimate of effect size when all observations required by the study protocol have been obtained [1, 2], but difficulties can arise if some observations are missing. Missing outcome observations can create a specific and often considerable challenge for the statistical analysis Handled, they can result in biased and inefficient estimates of effect size, threatening the intrinsic strength of the RCT design and compromising the ability to draw valid inferences from the study findings [3]. A considerable number of statistical methods have been, and continue to be, proposed for handling missing observations, but as yet universally accepted robust methods for handling missing data in RCTs do not exist [5]

Methods

Results

Discussion

Conclusion