Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals

Abstract

Randomized controlled trials almost always have some individuals with missing outcomes. Inadequate handling of these missing data in the analysis can cause substantial bias in the treatment effect estimates. We examine how missing outcome data are handled in randomized controlled trials in order to assess whether adequate steps have been taken to reduce nonresponse bias and to identify ways to improve procedures for missing data. We reviewed all randomized trials published between July and December 2001 in BMJ, JAMA, Lancet and New England Journal of Medicine, excluding trials in which the primary outcome was described as a time-to-event. We focused on trial designs, how missing outcome data were described and the statistical methods used to deal with the missing outcome data, including sensitivity analyses. We identified 71 trials of which 63 (89%) reported having partly missing outcome data: 13 trials had more than 20% of patients with missing outcomes. In 26 trials that measured the outcome at a single time point, 92% performed a complete case analysis and 8% imputed the missing outcomes using baseline values or the worst case value. In 37 trials with repeated measures of the outcome, 46% performed complete case analyses, potentially excluding individuals with some follow-up data, while 14% performed a repeated measures analysis, 19% used the last observation carried forward, 11% imputed with the worst case value and 2% imputed using regression predictions. Thirteen (21%) of trials with missing data reported a sensitivity analysis. Our review shows that missing outcome data are a common problem in randomized controlled trials, and are often inadequately handled in the statistical analysis in the top tier medical journals. Authors should explicitly state the assumptions underlying the handling of the missing outcomes and justify them through data descriptions and sensitivity analyses.