A review of the use of controlled multiple imputation in randomised controlled trials with missing outcome data

Ping-Tee Tan,Suzie Cro,Eleanor Van Vogt,Victoria R. Cornelius,Matyas Szigeti

doi:10.1186/s12874-021-01261-6

Ping-Tee Tan, Suzie Cro + Show 3 more

Open Access

https://doi.org/10.1186/s12874-021-01261-6

Copy DOI

Abstract

BackgroundMissing data are common in randomised controlled trials (RCTs) and can bias results if not handled appropriately. A statistically valid analysis under the primary missing-data assumptions should be conducted, followed by sensitivity analysis under alternative justified assumptions to assess the robustness of results. Controlled Multiple Imputation (MI) procedures, including delta-based and reference-based approaches, have been developed for analysis under missing-not-at-random assumptions. However, it is unclear how often these methods are used, how they are reported, and what their impact is on trial results. This review evaluates the current use and reporting of MI and controlled MI in RCTs.MethodsA targeted review of phase II-IV RCTs (non-cluster randomised) published in two leading general medical journals (The Lancet and New England Journal of Medicine) between January 2014 and December 2019 using MI. Data was extracted on imputation methods, analysis status, and reporting of results. Results of primary and sensitivity analyses for trials using controlled MI analyses were compared.ResultsA total of 118 RCTs (9% of published RCTs) used some form of MI. MI under missing-at-random was used in 110 trials; this was for primary analysis in 43/118 (36%), and in sensitivity analysis for 70/118 (59%) (3 used in both). Sixteen studies performed controlled MI (1.3% of published RCTs), either with a delta-based (n = 9) or reference-based approach (n = 7). Controlled MI was mostly used in sensitivity analysis (n = 14/16). Two trials used controlled MI for primary analysis, including one reporting no sensitivity analysis whilst the other reported similar results without imputation. Of the 14 trials using controlled MI in sensitivity analysis, 12 yielded comparable results to the primary analysis whereas 2 demonstrated contradicting results. Only 5/110 (5%) trials using missing-at-random MI and 5/16 (31%) trials using controlled MI reported complete details on MI methods.ConclusionsControlled MI enabled the impact of accessible contextually relevant missing data assumptions to be examined on trial results. The use of controlled MI is increasing but is still infrequent and poorly reported where used. There is a need for improved reporting on the implementation of MI analyses and choice of controlled MI parameters.

Highlights

Missing data are common in randomised controlled trials (RCTs) and can bias results if not handled appropriately
There were 67 studies in NEJM and 51 studies in the Lancet that included the use of Multiple Imputation (MI) and/or controlled MI, which encompassed 9% from a total of 1267 RCTs published in both journals over the period, see Fig. 2
Performed diagnostic check of imputations a9 trials did not include all variables in the analysis model in the imputation model and included auxiliary variables. bOne trial specified that the Multiple Imputation-Automatic method was used. c Explicitly stated (n = 18) or inferable from specified software or reference (n = 7). dOne trial reported presented the overall 95% confidence using the mean of the values for the lower and upper confidence intervals

Summary

Introduction

Missing data are common in randomised controlled trials (RCTs) and can bias results if not handled appropriately. A statistically valid analysis under the primary missing-data assumptions should be conducted, followed by sensitivity analysis under alternative justified assumptions to assess the robustness of results. Controlled Multiple Imputation (MI) procedures, including delta-based and reference-based approaches, have been developed for analysis under missing-not-at-random assumptions. It is unclear how often these methods are used, how they are reported, and what their impact is on trial results. Missing data can compromise the credibility of the trial conclusions, especially when the amount is substantial [3]. Sensitivity analysis under alternative plausible missing data assumptions should be performed to assess the robustness of the trial results.

Methods

Results

Discussion

Conclusion