Multiple Imputation Estimation Research Articles

Comparison between inverse-probability weighting and multiple imputation in Cox model with missing failure subtype.

Identifying and distinguishing risk factors for heterogeneous disease subtypes has been of great interest. However, missingness in disease subtypes is a common problem in those data analyses. Several methods have been proposed to deal with the missing data, including complete-case analysis, inverse-probability weighting, and multiple imputation. Although extant literature has compared these methods in missing problems, none has focused on the competing risk setting. In this paper, we discuss the assumptions required when complete-case analysis, inverse-probability weighting, and multiple imputation are used to deal with the missing failure subtype problem, focusing on how to implement these methods under various realistic scenarios in competing risk settings. Besides, we compare these three methods regarding their biases, efficiency, and robustness to model misspecifications using simulation studies. Our results show that complete-case analysis can be seriously biased when the missing completely at random assumption does not hold. Inverse-probability weighting and multiple imputation estimators are valid when we correctly specify the corresponding models for missingness and for imputation, and multiple imputation typically shows higher efficiency than inverse-probability weighting. However, in real-world studies, building imputation models for the missing subtypes can be more challenging than building missingness models. In that case, inverse-probability weighting could be preferred for its easy usage. We also propose two automated model selection procedures and demonstrate their usage in a study of the association between smoking and colorectal cancer subtypes in the Nurses' Health Study and Health Professional Follow-Up Study.

Multiple imputation of missing data under missing at random: including a collider as an auxiliary variable in the imputation model can induce bias.

Epidemiological studies often have missing data, which are commonly handled by multiple imputation (MI). In MI, in addition to those required for the substantive analysis, imputation models often include other variables ("auxiliary variables"). Auxiliary variables that predict the partially observed variables can reduce the standard error (SE) of the MI estimator and, if they also predict the probability that data are missing, reduce bias due to data being missing not at random. However, guidance for choosing auxiliary variables is lacking. We examine the consequences of a poorly chosen auxiliary variable: if it shares a common cause with the partially observed variable and the probability that it is missing (i.e., it is a "collider"), its inclusion can induce bias in the MI estimator and may increase the SE. We quantify, both algebraically and by simulation, the magnitude of bias and SE when either the exposure or outcome is incomplete. When the substantive analysis outcome is partially observed, the bias can be substantial, relative to the magnitude of the exposure coefficient. In settings in which a complete records analysis is valid, the bias is smaller when the exposure is partially observed. However, bias can be larger if the outcome also causes missingness in the exposure. When using MI, it is important to examine, through a combination of data exploration and considering plausible casual diagrams and missingness mechanisms, whether potential auxiliary variables are colliders.

Multiple imputation of missing data under missing at random: compatible imputation models are not sufficient to avoid bias if they are mis-specified

ObjectivesEpidemiological studies often have missing data, which are commonly handled by multiple imputation (MI). Standard (default) MI procedures use simple linear covariate functions in the imputation model. We examine the bias that may be caused by acceptance of this default option and evaluate methods to identify problematic imputation models, providing practical guidance for researchers. Study Design and SettingUsing simulation and real data analysis, we investigated how imputation model mis-specification affected MI performance, comparing results with complete records analysis (CRA). We considered scenarios in which imputation model mis-specification occurred because (i) the analysis model was mis-specified or (ii) the relationship between exposure and confounder was mis-specified. ResultsMis-specification of the relationship between outcome and exposure, or between exposure and confounder, can cause biased CRA and MI estimates (in addition to any bias in the full-data estimate due to analysis model mis-specification). MI by predictive mean matching can mitigate model mis-specification. Methods for examining model mis-specification were effective in identifying mis-specified relationships. ConclusionWhen using MI methods that assume data are MAR, compatibility between the analysis and imputation models is necessary, but not sufficient to avoid bias. We propose a step-by-step procedure for identifying and correcting mis-specification of imputation models.

A Bootstrap Method for a Multiple-Imputation Variance Estimator in Survey Sampling

Rubin’s variance estimator of the multiple imputation estimator for a domain mean is not asymptotically unbiased. Kim et al. derived the closed-form bias for Rubin’s variance estimator. In addition, they proposed an asymptotically unbiased variance estimator for the multiple imputation estimator when the imputed values can be written as a linear function of the observed values. However, this needs the assumption that the covariance of the imputed values in the same imputed dataset is twice that in the different imputed datasets. In this study, we proposed a bootstrap variance estimator that does not need this assumption. Both theoretical argument and simulation studies show that it was unbiased and asymptotically valid. The new method was applied to the Hox pupil popularity data for illustration.

Unified estimation for Cox regression model with nonmonotone missing at random covariates.

This article investigates a unified estimator for Cox regression model (Cox, 1972) when covariate data are missing at random (Rubin, 1976). It extends the idea of using parametric working models (Zhao and Liu, 2021) to extract the partial information contained in the incomplete observations. The working models are flexible and convenient to deal with nonmonotone missing data patterns. It can also incorporate auxiliary variables into the analysis to reduce estimation bias and improve efficiency. The unified estimator is consistent and more efficient than the (weighted) complete case estimator. Similar to multiple imputation (MI) method (Rubin, 1987 and 1996), the proposed method is also based on standard (weighted) complete data analysis and can be easily implemented in standard software. Simulation studies comparing the unified estimator with the substantive model compatible modification of the fully conditional specification MI (SMC-FCS) estimator (Bartlett et al., 2015) in various settings indicate that the unified estimator is consistent and as efficient as SMC-FCS estimator. Data from a clinical trial in patients with early breast cancer are analyzed for illustration.

Comparison of missing data handling methods in cognitive diagnosis: Zero replacement, multiple imputation and maximum likelihood estimation

Comparison of missing data handling methods in cognitive diagnosis: Zero replacement, multiple imputation and maximum likelihood estimation

Goodness-of-fit tests for a logistic regression model with missing covariates

Model checking for logistic regression with covariates missing at random is considered. Based on the ideas of Copas (1989) and Osius and Rojek (1992) and studies of Homser et al. (1997), proposed are the two-type goodness-of-fit tests, Pearson chi-squared and unweighted residual sum-of-squares tests, in which their test statistics are centralized by subtracting their estimated mean to be mean-zero-form test statistics via the inverse probability weighting (IPW) and nonparametric multiple imputation (MI) methods to solve the missing value problem. The asymptotic properties of these test statistics are established under the null hypothesis and some regularity conditions. The test statistics conducted by using the IPW and MI estimators are asymptotically equivalent. Proposed are the IPW method and two bootstrap re-sampling approaches for estimation of the variances of the proposed test statistics to solve the issue of underestimating their variances by the MI method of Rubin (1987). Simulation studies are carried out to assess the finite-sample power performances of these proposed tests. Two real data examples are used to illustrate the applicability of the proposed tests.

Sensitivity to missing not at random dropout in clinical trials: Use and interpretation of the trimmed means estimator.

Outcome values in randomized controlled trials (RCTs) may be missing not at random (MNAR), if patients with extreme outcome values are more likely to drop out (eg, due to perceived ineffectiveness of treatment, or adverse effects). In such scenarios, estimates from complete case analysis (CCA) and multiple imputation (MI) will be biased. We investigate the use of the trimmed means (TM) estimator for the case of univariable missingness in one continuous outcome. The TM estimator operates by setting missing values to the most extreme value, and then “trimming” away equal fractions of both groups, estimating the treatment effect using the remaining data. The TM estimator relies on two assumptions, which we term the “strong MNAR” and “location shift” assumptions. We derive formulae for the TM estimator bias resulting from the violation of these assumptions for normally distributed outcomes. We propose an adjusted TM estimator, which relaxes the location shift assumption and detail how our bias formulae can be used to establish the direction of bias of CCA and TM estimates, to inform sensitivity analyses. The TM approach is illustrated in a sensitivity analysis of the CoBalT RCT of cognitive behavioral therapy (CBT) in 469 individuals with 46 months follow‐up. Results were consistent with a beneficial CBT treatment effect, with MI estimates closer to the null and TM estimates further from the null than the CCA estimate. We propose using the TM estimator as a sensitivity analysis for data where extreme outcome value dropout is plausible.

Combining multiple imputation with raking of weights: Anefficient and robust approach in the setting of nearly true models.

Multiple imputation (MI) provides us with efficient estimators in model-based methods for handling missing data under the true model. It is also well-understood that design-based estimators are robust methods that do not require accurately modeling the missing data; however, they can be inefficient. In any applied setting, it is difficult to know whether a missing data model may be good enough to win the bias-efficiency trade-off. Raking of weights is one approach that relies on constructing an auxiliary variable from data observed on the full cohort, which is then used to adjust the weights for the usual Horvitz-Thompson estimator. Computing the optimally efficient raking estimator requires evaluating the expectation of the efficient score given the full cohort data, which is generally infeasible. We demonstrate MI as a practical method to compute a raking estimator that will be optimal. We compare this estimator to common parametric and semi-parametric estimators, including standard MI. We show that while estimators, such as the semi-parametric maximum likelihood and MI estimator, obtain optimal performance under the true model, the proposed raking estimator utilizing MI maintains a better robustness-efficiency trade-off even under mild model misspecification. We also show that the standard raking estimator, without MI, is often competitive with the optimal raking estimator. We demonstrate these properties through several numerical examples and provide a theoretical discussion of conditions for asymptotically superior relative efficiency of the proposed raking estimator.

SMIM: A unified framework of survival sensitivity analysis using multiple imputation and martingale.

Censored survival data are common in clinical trial studies. We propose a unified framework for sensitivity analysis to censoring at random in survival data using multiple imputation and martingale, called SMIM. The proposed framework adopts the δ-adjusted and control-based models, indexed by the sensitivity parameter, entailing censoring at random and a wide collection of censoring not at random assumptions. Also, it targets a broad class of treatment effect estimands defined as functionals of treatment-specific survival functions, taking into account missing data due to censoring. Multiple imputation facilitates the use of simple full-sample estimation; however, the standard Rubin's combining rule may overestimate the variance for inference in the sensitivity analysis framework. We decompose the multiple imputation estimator into a martingale series based on the sequential construction of the estimator and propose the wild bootstrap inference by resampling the martingale series. The new bootstrap inference has a theoretical guarantee for consistency and is computationally efficient compared to the nonparametric bootstrap counterpart. We evaluate the finite-sample performance of the proposed SMIM through simulation and an application on an HIV clinicaltrial.

A comparison of full information maximum likelihood and multiple imputation in structural equation modeling with missing data.

This article compares two missing data procedures, full information maximum likelihood (FIML) and multiple imputation (MI), to investigate their relative performances in relation to the results from analyses of the original complete data or the hypothetical data available before missingness occurred. By expressing the FIML estimator as a special MI estimator, we predicted the expected patterns of discrepancy between the two estimators. Via Monte Carlo simulation studies where we have access to the original complete data, we compare the performance of FIML and MI estimators to that of the complete data maximum likelihood (ML) estimator under a wide range of conditions, including differences in sample size, percent of missingness, and degrees of model misfit. Our study confirmed well-known knowledge that the two estimators tend to yield essentially equivalent results to each other and to those from analysis of complete data when the postulated model is correctly specified. However, some noteworthy patterns of discrepancies were found between the FIML and MI estimators when the hypothesized model does not hold exactly in the population: MI-based parameter estimates, comparative fit index (CFI), and the Tucker Lewis index (TLI) tend to be closer to the counterparts of the complete data ML estimates, whereas FIML-based chi-squares and root mean square error of approximation (RMSEA) tend to be closer to the counterparts of the complete data ML estimates. We explained the observed patterns of discrepancy between the two estimators as a function of the interplay between the parsimony and accuracy of the imputation model. We concluded by discussing practical and methodological implications and issues for further research. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Selecting the model for multiple imputation of missing data: Just use an IC!

Multiple imputation and maximum likelihood estimation (via the expectation‐maximization algorithm) are two well‐known methods readily used for analyzing data with missing values. While these two methods are often considered as being distinct from one another, multiple imputation (when using improper imputation) is actually equivalent to a stochastic expectation‐maximization approximation to the likelihood. In this article, we exploit this key result to show that familiar likelihood‐based approaches to model selection, such as Akaike's information criterion (AIC) and the Bayesian information criterion (BIC), can be used to choose the imputation model that best fits the observed data. Poor choice of imputation model is known to bias inference, and while sensitivity analysis has often been used to explore the implications of different imputation models, we show that the data can be used to choose an appropriate imputation model via conventional model selection tools. We show that BIC can be consistent for selecting the correct imputation model in the presence of missing data. We verify these results empirically through simulation studies, and demonstrate their practicality on two classical missing data examples. An interesting result we saw in simulations was that not only can parameter estimates be biased by misspecifying the imputation model, but also by overfitting the imputation model. This emphasizes the importance of using model selection not just to choose the appropriate type of imputation model, but also to decide on the appropriate level of imputation model complexity.

Validation likelihood estimation method for a zero-inflated Bernoulli regression model with missing covariates

A zero-inflated Bernoulli (ZIBer) regression model is an alternative model in order to improve fitting the binary outcome data that have many more zeros than expected under a regular logistic regression model. Because some covariates often have missing values, we propose the validation likelihood (VL) method to estimate the parameters of the ZIBer regression model with covariates missing at random. We consider the true selection probability (SP) model, a parametric SP model and a nonparametric SP model to compare the efficiency of the true SP, parametric SP, and semiparametric SP VL estimators. The asymptotic results of these three estimators are studied under some regularity conditions. Both the theoretical and simulation results reveal that the parametric and semiparametric SP VL estimation methods outperform the true SP VL estimation method if the parametric SP model is correct. The simulation results also indicate that the semiparametric SP VL estimation method is preferred to the inverse probability weighting and multiple imputation estimation methods. The practical application of the proposed methodology is adorned with a survey data set of violations of traffic rules of motorcyclist respondents in Taiwan.

Response Surface Analysis with Missing Data

Response Surface Analysis (RSA) is gaining popularity in psychological research as a tool for investigating congruence hypotheses (e.g., consequences of self-other agreement, person-job fit, dyadic similarity). RSA involves the estimation of a nonlinear polynomial regression model and the interpretation of the resulting response surface. However, little is known about how best to conduct RSA when the underlying data are incomplete. In this article, we compare different methods for handling missing data in RSA. This includes different strategies for multiple imputation (MI) and maximum-likelihood (ML) estimation. Specifically, we consider the “just another variable” (JAV) approach to MI and ML, an approach that is in regular use in applications of RSA, and the more novel “substantive-model-compatible” (SMC) approach. In a simulation study, we evaluate the impact of these methods on focal outcomes of RSA, including the accuracy of parameter estimates, the shape of the response surface, and the testing of congruence hypotheses. Our findings suggest that the JAV approach can sometimes distort parameter estimates and conclusions about the shape of the response surface, whereas the SMC approach performs well overall. We illustrate applications of the methods in a worked example with real data and provide recommendations for their application in practice.

Multiple imputation and direct estimation for qPCR data with non-detects

BackgroundQuantitative real-time PCR (qPCR) is one of the most widely used methods to measure gene expression. An important aspect of qPCR data that has been largely ignored is the presence of non-detects: reactions failing to exceed the quantification threshold and therefore lacking a measurement of expression. While most current software replaces these non-detects with a value representing the limit of detection, this introduces substantial bias in the estimation of both absolute and differential expression. Single imputation procedures, while an improvement on previously used methods, underestimate residual variance, which can lead to anti-conservative inference.ResultsWe propose to treat non-detects as non-random missing data, model the missing data mechanism, and use this model to impute missing values or obtain direct estimates of model parameters. To account for the uncertainty inherent in the imputation, we propose a multiple imputation procedure, which provides a set of plausible values for each non-detect. We assess the proposed methods via simulation studies and demonstrate the applicability of these methods to three experimental data sets. We compare our methods to mean imputation, single imputation, and a penalized EM algorithm incorporating non-random missingness (PEMM). The developed methods are implemented in the R/Bioconductor package nondetects.ConclusionsThe statistical methods introduced here reduce discrepancies in gene expression values derived from qPCR experiments in the presence of non-detects, providing increased confidence in downstream analyses.

Handling Missing Data in the Short Form-12 Health Survey (SF-12): Concordance of Real Patient Data and Data Estimated by Missing Data Imputation Procedures.

If information on single items in the Short Form–12 health survey (SF-12) is missing, the analysis of only complete cases causes a loss of statistical power and, in case of nonrandom missing data (MD), systematic bias. This study aimed at evaluating the concordance of real patient data and data estimated by different MD imputation procedures in the items of the SF-12 assessment. For this ends, MD were examined in a sample of 1,137 orthopedic patients. Additionally, MD were simulated (a) in the subsample of orthopedic patients exhibiting no MD (n = 810; 71%) as well as (b) in a sample of 6,970 respondents representing the German general population (95.8% participants with complete data) using logistic regression modelling. Simulated MD were replaced by mean values as well as regression-, expectation-maximization- (EM-), and multiple imputation estimates. Higher age and lower education were associated with enhanced probabilities of MD. In terms of accuracy in both data sets, the EM-procedure (ICC2,1 = .33-.72) outperformed alternative estimation approaches substantially (e.g., regression imputation: ICC2,1 = .18-.48). The EM-algorithm can be recommended to estimate MD in the items of the SF-12, because it reproduces the actual patient data most accurately.

An Alternative Implementation of Reference-Based Controlled Imputation Procedures

Reference-based controlled imputations have become a popular tool to assess the sensitivity of primary analysis inference to different post-dropout assumptions and to yield an alternative effectiveness estimand of treatment effect. As the imputation and analysis models are uncongenial in this setting, Rubin’s variance estimator overestimates the repeated sampling variability of the multiple imputation estimator of treatment effect. On the other hand, since reference-based methods borrow information across arms, the sampling variance of treatment effect estimator decreases as the proportion of missing data increases, rendering it inappropriate for use in sensitivity analysis. We propose to decouple the variance estimation from information borrowing by using a delta-adjusted pattern mixture model with delta-adjustments fixed at the maximum likelihood estimate. Reference-based pattern-mixture models can thus be embedded in the sensitivity analysis using delta-adjusted pattern-mixture models by identifying a scalar sensitivity parameter value to match the treatment effect estimate. We provide both theoretical and empirical justifications of the proposed approach and illustrate its use in the analysis of a clinical trial of major depressive disorder.

Accounting for model uncertainty in multiple imputation under complex sampling

AbstractMultiple imputation provides an effective way to handle missing data. When several possible models are under consideration for the data, multiple imputation is typically performed under a single‐best model selected from the candidate models. This single‐model selection approach ignores the uncertainty associated with the model selection and so leads to underestimation of the variance of multiple imputation estimator. In this article, we propose a new multiple imputation procedure incorporating model uncertainty in the final inference. The proposed method incorporates possible candidate models for the data into the imputation procedure using the idea of Bayesian model averaging. The proposed method is directly applicable to handling item nonresponse in survey sampling. Asymptotic properties of the proposed method are investigated. A limited simulation study confirms that our model averaging approach provides better estimation performance than the single‐model selection approach.

Handling missing data in modelling quality of clinician-prescribed routine care: Sensitivity analysis of departure from missing at random assumption.

Missing information is a major drawback in analyzing data collected in many routine health care settings. Multiple imputation assuming a missing at random mechanism is a popular method to handle missing data. The missing at random assumption cannot be confirmed from the observed data alone, hence the need for sensitivity analysis to assess robustness of inference. However, sensitivity analysis is rarely conducted and reported in practice. We analyzed routine paediatric data collected during a cluster randomized trial conducted in Kenyan hospitals. We imputed missing patient and clinician-level variables assuming the missing at random mechanism. We also imputed missing clinician-level variables assuming a missing not at random mechanism. We incorporated opinions from 15 clinical experts in the form of prior distributions and shift parameters in the delta adjustment method. An interaction between trial intervention arm and follow-up time, hospital, clinician and patient-level factors were included in a proportional odds random-effects analysis model. We performed these analyses using R functions derived from the jomo package. Parameter estimates from multiple imputation under the missing at random mechanism were similar to multiple imputation estimates assuming the missing not at random mechanism. Our inferences were insensitive to departures from the missing at random assumption using either the prior distributions or shift parameters sensitivity analysis approach.

PDB34 THE DIABETES PREVENTION PROGRAM (DPP) QUALITY OF WELL BEING (QWB) UTILITY MEASURES AND MISSING DATA CONSIDERATIONS: AVAILABLE CASE ANALYSIS (ACA) VS. MULTIPLE IMPUTATION (MI) RESULTS FOR UTILITIES AND QUALITY ADJUSTED LIFE YEARS (QALYS)

In 2003 the DPP published a within-trial 3 year cost-effectiveness analysis (CEA). The authors appear to have ignored missing data and used ACA, reporting QALYs using the QWB to measure patient utilities for placebo (PBO), lifestyle (LS), and metformin (MET). The main objective of this research is to estimate treatment utilities and QALYs using Multiple Imputation (MI) and compare these results with previous ACA findings. We had access to a 95% subset of the original data. We reproduced the original ACA results and then used MI to develop estimates for utilities and QALYs based on the QALY calculation method used in the original CEA. 5 imputed datasets were pooled into a single imputed dataset. The percentages of participants with at least one missing utility data for QWB for years 1, 2 and 3 were 36%, 10 % and 48%. The yearly utility scores using the QWB originally reported in the DPP were [0.686, 0.675, 0.657], [0.687, 0.680, 0.673] and [0.703, 0.695, 0.692] for PBO, MET, and LS. The utility scores using MI were [0.685, 0.676, 0.669], [0.688, 0.683, 0.681] and [0.705, 0.694, 0.694]. QALYs using the QWB as originally calculated were 2.018, 2.040 and 2.090 for PBO, MET and LS. The MI QALYs were 2.030, 2.053 and 2.094. The incremental QALYs using the QWB as originally calculated were 0.072, 0.022 and 0.050 for LS versus PBO, MET versus PBO, and LS versus MET. The MI incremental QALYs were 0.064, 0.023 and 0.041. The missing data and MI estimates in the DPP yielded different incremental QALYs compared to the ones reported in the original DPP study. Small differences in QALYs have been shown previously to be important in the DPP CEA conclusions. These differences may be important in their CEA implications, a subject for future research.