An empirical comparison of Bayesian modelling strategies for missing binary outcome data in network meta-analysis

Loukia M Spineli

doi:10.1186/s12874-019-0731-y

Abstract

BackgroundA number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews. However, none of these have been evaluated empirically in a series of published systematic reviews.MethodsUsing published systematic reviews with network meta-analysis (NMA) from a wide range of health-related fields, we evaluated comparatively the most frequently described Bayesian modelling strategies for MOD in terms of log odds ratio (log OR), between-trial variance, inconsistency factor (i.e. difference between direct and indirect estimates for a comparison), surface under the cumulative ranking (SUCRA) and rankings. We extended the Bayesian random-effects NMA model to incorporate the informative missingness odds ratio (IMOR) parameter, and applied the node-splitting approach to investigate inconsistency locally. We considered both pattern-mixture and selection models, different structures for prior distribution of log IMOR, and different scenarios for MOD. To illustrate level of agreement between different strategies and scenarios, we used Bland-Altman plots.ResultsAddressing MOD using extreme scenarios and ignoring the uncertainty about the scenarios led to systematically different and more precise log ORs compared to modelling MOD under the missing at random (MAR) assumption. Hierarchical structure of log IMORs led to lower between-trial variance, especially in the case of substantial MOD. Assuming common-within-network or trial-specific log IMORs yielded similar posterior results for all NMA estimates, whereas intervention-specific structure systematically inflated uncertainty around log ORs and SUCRAs. Pattern-mixture model agreed with selection model, particularly under the trial-specific structure; however, selection model systematically reduced precision around log IMORs. Overall, different strategies and scenarios mostly had good agreement in the case of low MOD.ConclusionsAddressing MOD using extreme scenarios and/or ignoring the uncertainty about the scenarios may negatively affect NMA estimates. Modelling MOD via the IMOR parameter can ensure bias-adjusted estimates and offer valuable insights into missingness mechanisms. The researcher should seek an expert opinion in order to decide on the structure of log IMOR that best aligns to the condition and interventions studied and to define a proper prior distribution for log IMOR. Our findings also apply to pairwise meta-analyses.

Highlights

A number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews
Implications of extreme scenarios about the missingness mechanism Overall, differences in terms of posterior mean of log odds ratio (OR) ranged in much narrower limits of agreement (LoA) for on average missing at random (MAR) versus more missing cases are events (MME) and more missing cases are non-events (MMNE) as opposed to on average MAR versus Best-case scenario for all non-reference interventions (BC) and Worst-case scenario for all non-reference interventions (WC) where almost all differences were concentrated systematically below and above 0, respectively, for networks with moderate and large MOD (Fig. 2)
We found that consideration of BC or WC resulted systematically in much larger and lower log odds ratio (log OR), respectively, when the network was predominated by trials with moderate or large MOD (Fig. 2)

Summary

Introduction

A number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews. A handful of these methodologies have been extended further to operate in a network of several interventions [8, 11] These methodological articles provide only limited empirical evidence to demonstrate the merits and demerits of proposed methods as they usually consider one published systematic review with pairwise or network meta-analyses (NMA). When included trials provide limited or no information on the reasons for MOD, in order to explore assumptions empirically, the meta-analyst examines the sensitivity of results to plausible scenarios [2]. A usual starting point of the analysis is to assume that data are missing at random (MAR) and investigate any deviations from this assumption by performing a series of sensitivity analyses (Additional file 1: Table S1) [2,3,4, 12]

Methods

Results

Discussion

Conclusion