Missing Data Mechanism Research Articles

Treatment Benefit and Treatment Harm Rates with Nonignorable Missing Covariate, Endpoint, or Treatment

The average treatment effect is an important concept in causal inference. However, it fails to capture variation in response to treatment due to heterogeneity at many levels among patients in the target population. To study the heterogeneity in the treatment effect, researchers proposed the concepts of treatment benefit rate (TBR) and treatment harm rate (THR). Howerver, in practice, missing data often occurs in treatment, endpoints, and covariates. In these cases, the conditions given by them are not enough to identify treatment benefit rate. In this article, we address the problem of identifying the treatment benefit rate and treatment harm rate when treatment or endpoints or covariates are missing. Different types of missing data mechanisms are assumed, including several situations of nonignorable missingness. We prove that the treatment benefit rate and treatment harm rate are identifiable under very mild conditions, and then construct estimators based on methods of the EM algorithm. The performance of the proposed inference procedure is evaluated via simulation studies. Lastly, we illustrate our method by real data sets.

Verification Bias Correction in Endometrial Abnormalities in Infertile Women Referred to Royan Institute Using Statistical Methods.

Verification bias is a common bias in the diagnostic accuracy of diagnostic tests and occurs when a number of individuals do not perform the gold standard test. In this study, we review the correcting methods of verification bias. In a cross-sectional study in 2020, 567 infertile women who were referred to Royan Research Institute were evaluated. The ultrasound is the performed test and the gold standard are hysteroscopy for some, and pathology for other abnormalities. For correcting verification bias conventional, Begg and Greens, Zhou, and logistic regression methods were used. In the gold standard hysteroscopy test, the sensitivity (SEN) and specificity (SPEC) obtained in conventional, Begg and Greens, Zhou, and logistics Regression methods were (50%, 90.3%), (48%, 96%), (22%, 77%), (50%, 90%), and (72.8, 77) respectively. Furthermore, the area under the curve (AUC) index and kappa statistics were calculated as 70.2%, and 43.6% respectively. In the pathology gold standard test, the SEN and SPEC for the conventional methods, Begg and Greens, Zhou and logistics regression were (67.7%, 86.7%), (66%, 88%), (29%, 70%), (66.9%, 87.6%), and (73%, 83.9%) respectively. Also, the AUC index and kappa statistics were 77%, and 55% respectively. In the study on endometrial abnormalities in infertile women, assuming that the missing data mechanism is random, the amount of bias in calculating SEN and SPEC is very low in the diagnostic tests calculated before and after correction, using Begg and Greens and logistic regression method. But Zhou's method gives rather large biased estimates.

Randomization-Based Inference for Clinical Trials with Missing Outcome Data

Randomization-based inference is a natural way to analyze data from a clinical trial. But the presence of missing outcome data is problematic: if the data are removed, the randomization distribution is destroyed and randomization tests have no validity. In this article we describe two approaches to imputing values for missing data that preserve the randomization distribution. We then compare these methods to population-based and parametric imputation approaches that are in standard use to compare error rates under both homogeneous and heterogeneous population models. We also describe randomization-based analogs of standard missing data mechanisms and describe a randomization-based procedure to determine if data are missing completely at random. We conclude that randomization-based methods are a reasonable approach to missing data that perform comparably to population-based methods.

A Machine Learning-Based Multiple Imputation Method for the Health and Aging Brain Study–Health Disparities

The Health and Aging Brain Study–Health Disparities (HABS–HD) project seeks to understand the biological, social, and environmental factors that impact brain aging among diverse communities. A common issue for HABS–HD is missing data. It is impossible to achieve accurate machine learning (ML) if data contain missing values. Therefore, developing a new imputation methodology has become an urgent task for HABS–HD. The three missing data assumptions, (1) missing completely at random (MCAR), (2) missing at random (MAR), and (3) missing not at random (MNAR), necessitate distinct imputation approaches for each mechanism of missingness. Several popular imputation methods, including listwise deletion, min, mean, predictive mean matching (PMM), classification and regression trees (CART), and missForest, may result in biased outcomes and reduced statistical power when applied to downstream analyses such as testing hypotheses related to clinical variables or utilizing machine learning to predict AD or MCI. Moreover, these commonly used imputation techniques can produce unreliable estimates of missing values if they do not account for the missingness mechanisms or if there is an inconsistency between the imputation method and the missing data mechanism in HABS–HD. Therefore, we proposed a three-step workflow to handle missing data in HABS–HD: (1) missing data evaluation, (2) imputation, and (3) imputation evaluation. First, we explored the missingness in HABS–HD. Then, we developed a machine learning-based multiple imputation method (MLMI) for imputing missing values. We built four ML-based imputation models (support vector machine (SVM), random forest (RF), extreme gradient boosting (XGB), and lasso and elastic-net regularized generalized linear model (GLMNET)) and adapted the four ML-based models to multiple imputations using the simple averaging method. Lastly, we evaluated and compared MLMI with other common methods. Our results showed that the three-step workflow worked well for handling missing values in HABS–HD and the ML-based multiple imputation method outperformed other common methods in terms of prediction performance and change in distribution and correlation. The choice of missing handling methodology has a significant impact on the accompanying statistical analyses of HABS–HD. The conceptual three-step workflow and the ML-based multiple imputation method perform well for our Alzheimer’s disease models. They can also be applied to other disease data analyses.

Semiparametric multivariate joint model for skewed-longitudinal and survival data: A Bayesian approach.

Joint models and statistical inference for longitudinal and survival data have been an active area of statistical research and have mostly coupled a longitudinal biomarker-based mixed-effects model with normal distribution and an event time-based survival model. In practice, however, the following issues may standout: (i) Normality of model error in longitudinal models is a routine assumption, but it may be unrealistically violating data features of subject variations. (ii) Data collected are often featured by the mixed types of multiple longitudinal outcomes which are significantly correlated, ignoring their correlation may lead to biased estimation. Additionally, a parametric model specification may be inflexible to capture the complicated patterns of longitudinal data. (iii) Missing observations in the longitudinal data are often encountered; the missing measures are likely to be informative (nonignorable) and ignoring this phenomenon may result in inaccurate inference. Multilevel item response theory (MLIRT) models have been increasingly used to analyze the multiple longitudinal data of mixed types (ie, continuous and categorical) in clinical studies. In this article, we develop an MLIRT-based semiparametric joint model with skew-t distribution that consists of an extended MLIRT model for the mixed types of multiple longitudinal data and a Cox proportional hazards model, linked through random-effects. A Bayesian approach is employed for joint modeling. Simulation studies are conducted to assess performance of the proposed models and method. A real example from primary biliary cirrhosis clinical study is analyzed to estimate parameters in the joint model and also evaluate sensitivity of parameter estimates for various plausible nonignorable missing data mechanisms.

Effect of Missing Data on Test Equating Methods Under NEAT Design

In this study, it was aimed to examine the effect of missing data in different patterns and sizes on test equating methods under the NEAT design for different factors. For this purpose, as part of this study, factors such as sample size, average difficulty level difference between the test forms, difference between the ability distribution, missing data rate, and missing data mechanisms were manipulated. The effects of these factors on the equating error of test equating methods (chained-equipercentile equating, Tucker, frequency estimation equating, and Braun-Holland) were investigated. In the study, two separate sets of 10,000 dichotomous data were generated consistent with a 2-parameter logistic model. While generating data, the MCAR and MAR missing data mechanisms were used. All analyses were conducted by R 4.2.2. As a result of the study, it was seen that the RMSE of the equating methods increased significantly as the missing data rate increased. The results indicate that the RMSE of the equating methods with imputed missing data are reduced compared to equating without imputed missing data. Furthermore, the percentage of missing data, along with the difference between ability levels and the average difficulty difference between forms, was found to significantly affect equating errors in the presence of missing data. Although increasing sample size did not have a significant effect on equating error in the presence of missing data, it did lead to more accurate equating when there was no missing data present.

Covariate adjustment in randomized clinical trials with missing covariate and outcome data.

When analyzing data from randomized clinical trials, covariate adjustment can be used to account for chance imbalance in baseline covariates and to increase precision of the treatment effect estimate. A practical barrier to covariate adjustment is the presence of missing data. In this article, in the light of recent theoretical advancement, we first review several covariate adjustment methods with incomplete covariate data. We investigate the implications of the missing data mechanism on estimating the average treatment effect in randomized clinical trials with continuous or binary outcomes. In parallel, we consider settings where the outcome data are fully observed or are missing at random; in the latter setting, we propose a full weighting approach that combines inverse probability weighting for adjusting missing outcomes and overlap weighting for covariate adjustment. We highlight the importance of including the interaction terms between the missingness indicators and covariates as predictors in the models. We conduct comprehensive simulation studies to examine the finite-sample performance of the proposed methods and compare with a range of common alternatives. We find that conducting the proposed adjustment methods generally improves the precision of treatment effect estimates regardless of the imputation methods when the adjusted covariate is associated with the outcome. We apply the methods to the Childhood Adenotonsillectomy Trial to assess the effect of adenotonsillectomy on neurocognitive functioning scores.

A novel missing data imputation approach based on clinical conditional Generative Adversarial Networks applied to EHR datasets

The missing data mechanism is a relevant problem in Machine Learning (ML) and biomedical informatics communities. Real-world Electronic Health Record (EHR) datasets comprise several missing values, thus revealing a high level of spatiotemporal sparsity in the predictors’ matrix. Several approaches in the state-of-the-art tried to deal with this problem by proposing different data imputation strategies that (i) are often unrelated to the ML model, (ii) are not conceived for EHR data where laboratory exams are not prescribed uniformly over time and percentage of missing values is high (iii) exploit only univariate and linear information on the observed features. Our paper proposes a data imputation strategy based on a clinical conditional Generative Adversarial Network (ccGAN) capable of imputing missing values by exploiting non-linear and multivariate information across patients. Unlike other GAN data imputation-based approaches, our method deals explicitly with the high level of missingness of routine EHR data by conditioning the imputing strategy to the observable values and those fully-annotated. We demonstrated the statistical significance of the ccGAN to other state-of-the-art approaches in terms of imputation (around 19.79% of gain to the best competitor) and predictive performance (up to 1.60% of gain to the best competitor) on a real multi-diabetic centers dataset. We also demonstrated its robustness across different missingness rates (up to 1.61% of gain to the best competitor in the highest missingness rates condition) on an additional benchmark EHR dataset.

Association Between Body Fat and Body Mass Index from Incomplete Longitudinal Proportion Data: Findings from the Fels Study

Obesity rates continue to exhibit an upward trajectory, particularly in the US, and is the underlying cause of several comorbidities, including but not limited to high blood pressure, high cholesterol, diabetes, heart disease, stroke, and cancers. To monitor obesity, body mass index (BMI) and proportion body fat (PBF) are two commonly used measurements. Although BMI and PBF changes over time in an individual’s lifespan and their relationship may also change dynamically, existing work has mostly remained cross-sectional, or separately modeling BMI and PBF. A combined longitudinal assessment is expected to be more effective in unravelling their complex interplay. To mitigate this, we consider Bayesian cross-domain latent growth curve models within a structural equation modeling framework, which simultaneously handles issues such as individually varying time metrics, proportion data, and potential missing not at random data for joint assessment of the longitudinal changes of BMI and PBF. Through simulation studies, we observe that our proposed models and estimation method yielded parameter estimates with small bias and mean squared error in general, however, a mis-specified missing data mechanism may cause inaccurate and inefficient parameter estimates. Furthermore, we demonstrate application of our method to a motivating longitudinal obesity study, controlling for both time-invariant (such as, sex), and time-varying (such as diastolic and systolic blood pressure, biceps skinfold, bioelectrical impedance, and waist circumference) covariates in separate models. Under time-invariance, we observe that the initial BMI level and the rate of change in BMI influenced PBF. However, in presence of time-varying covariates, only the initial BMI level influenced the initial PBF. The added-on selection model estimation indicated that observations with higher PBF values were less likely to be missing.

Missing data approaches in longitudinal studies of aging: A case example using the National Health and Aging Trends Study.

Missing data is a key methodological consideration in longitudinal studies of aging. We described missing data challenges and potential methodological solutions using a case example describing five-year frailty state transitions in a cohort of older adults. We used longitudinal data from the National Health and Aging Trends Study, a nationally-representative cohort of Medicare beneficiaries. We assessed the five components of the Fried frailty phenotype and classified frailty based on their number of components (robust: 0, prefrail: 1-2, frail: 3-5). One-, two-, and five-year frailty state transitions were defined as movements between frailty states or death. Missing frailty components were imputed using hot deck imputation. Inverse probability weights were used to account for potentially informative loss-to-follow-up. We conducted scenario analyses to test a range of assumptions related to missing data. Missing data were common for frailty components measured using physical assessments (walking speed, grip strength). At five years, 36% of individuals were lost-to-follow-up, differentially with respect to baseline frailty status. Assumptions for missing data mechanisms impacted inference regarding individuals improving or worsening in frailty. Missing data and loss-to-follow-up are common in longitudinal studies of aging. Robust epidemiologic methods can improve the rigor and interpretability of aging-related research.

Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study.

Background: In clinical prediction modelling, missing data can occur at any stage of the model pipeline; development, validation or deployment. Multiple imputation is often recommended yet challenging to apply at deployment; for example, the outcome cannot be in the imputation model, as recommended under multiple imputation. Regression imputation uses a fitted model to impute the predicted value of missing predictors from observed data, and could offer a pragmatic alternative at deployment. Moreover, the use of missing indicators has been proposed to handle informative missingness, but it is currently unknown how well this method performs in the context of clinical prediction models. Methods: We simulated data under various missing data mechanisms to compare the predictive performance of clinical prediction models developed using both imputation methods. We consider deployment scenarios where missing data is permitted or prohibited, imputation models that use or omit the outcome, and clinical prediction models that include or omit missing indicators. We assume that the missingness mechanism remains constant across the model pipeline. We also apply the proposed strategies to critical care data. Results: With complete data available at deployment, our findings were in line with existing recommendations; that the outcome should be used to impute development data when using multiple imputation and omitted under regression imputation. When missingness is allowed at deployment, omitting the outcome from the imputation model at the development was preferred. Missing indicators improved model performance in many cases but can be harmful under outcome-dependent missingness. Conclusion: We provide evidence that commonly taught principles of handling missing data via multiple imputation may not apply to clinical prediction models, particularly when data can be missing at deployment. We observed comparable predictive performance under multiple imputation and regression imputation. The performance of the missing data handling method must be evaluated on a study-by-study basis, and the most appropriate strategy for handling missing data at development should consider whether missing data are allowed at deployment. Some guidance is provided.

Updates in psoriasis diagnosis and treatment status in China: results from the National Psoriasis Center Registry.

Updates in psoriasis diagnosis and treatment status in China: results from the National Psoriasis Center Registry.

The Impact of Missing Data on the Performances of DIF Detection Methods

This study analyzed the impact of missing data techniques on performances of two differential item functioning (DIF) detection methods (Mantel Haenszel and Multiple Indicator and Multiple Causes) under missing completely at random missing data mechanism. Percentage of missing data was set at 5% and 15%. Zero imputation, listwise deletion and fractional hot-deck imputation were used to handle missing data. The data set of the study consisted of 17 items in the S12 item cluster of Programme for International Student Assessment (PISA) 2015 science test. Results showed that fractional hot-deck imputation produced the best results in identifying DIF items in all conditions and it had also the closest DIF values to the values obtained from complete data set. It was also found that multiple indicator and multiple causes method was more adversely affected than Mantel Haenszel by the presence of missing data.

Pay Attention to the Ignorable Missing Data Mechanisms! An Exploration of Their Impact on the Efficiency of Regression Coefficients

The use of modern missing data techniques has become more prevalent with their increasing accessibility in statistical software. These techniques focus on handling data that are missing at random (MAR). Although all MAR mechanisms are routinely treated as the same, they are not equal. The impact of missing data on the efficiency of parameter estimates can differ for different MAR variations, even when the amount of missing data is held constant; yet, in current practice, only the rate of missing data is reported. The impact of MAR on the loss of efficiency can instead be more directly measured by the fraction of missing information (FMI). In this article, we explore this impact using FMIs in regression models with one and two predictors. With the help of a Shiny application, we demonstrate that efficiency loss due to missing data can be highly complex and is not always intuitive. We recommend substantive researchers who work with missing data report estimates of FMIs in addition to the rate of missingness. We also encourage methodologists to examine FMIs when designing simulation studies with missing data, and to explore the behavior of efficiency loss under MAR using FMIs in more complex models.

Comparisons of statistical methods for handling attrition in a follow-up visit with complex survey sampling.

Design-based analysis, which accounts for the design features of the study, is commonly used to conduct data analysis in studies with complex survey sampling, such as the Hispanic Community Health Study/Study of Latinos (HCHS/SOL). In this type of longitudinal study, attrition has often been a problem. Although there have been various statistical approaches proposed to handle attrition, such as inverse probability weighting (IPW), non-response cell weighting (NRCW), multiple imputation (MI), and full information maximum likelihood (FIML) approach, there has not been a systematic assessment of these methods to compare their performance in design-based analyses. In this article, we perform extensive simulation studies and compare the performance of different missing data methods in linear and generalized linear population models, and under different missing data mechanism. We find that the design-based analysis is able to produce valid estimation and statistical inference when the missing data are handled appropriately using IPW, NRCW, MI, or FIML approach under missing-completely-at-random or missing-at-random missing mechanism and when the missingness model is correctly specified or over-specified. We also illustrate the use of these methods using data from HCHS/SOL.

Impute-then-exclude versus exclude-then-impute: Lessons when imputing a variable used both in cohort creation and as an independent variable in the analysis model.

We examined the setting in which a variable that is subject to missingness is used both as an inclusion/exclusion criterion for creating the analytic sample and subsequently as the primary exposure in the analysis model that is of scientific interest. An example is cancer stage, where patients with stage IV cancer are often excluded from the analytic sample, and cancer stage (I to III) is an exposure variable in the analysis model. We considered two analytic strategies. The first strategy, referred to as "exclude-then-impute," excludes subjects for whom the observed value of the target variable is equal to the specified value and then uses multiple imputation to complete the data in the resultant sample. The second strategy, referred to as "impute-then-exclude," first uses multiple imputation to complete the data and then excludes subjects based on the observed or filled-in values in the completed samples. Monte Carlo simulations were used to compare five methods (one based on "exclude-then-impute" and four based on "impute-then-exclude") along with the use of a complete case analysis. We considered both missing completely at random and missing at random missing data mechanisms. We found that an impute-then-exclude strategy using substantive model compatible fully conditional specification tended to have superior performance across 72 different scenarios. We illustrated the application of these methods using empirical data on patients hospitalized with heart failure when heart failure subtype was used for cohort creation (excluding subjects with heart failure with preserved ejection fraction) and was also an exposure in the analysis model.

How to test the missing data mechanism in a hidden Markov model

A Hidden Markov Model with missing data in the outcome variable is considered. The initial and transition probabilities of the Markov chain and the emission probability of the HMM are allowed to depend on fully observed covariables. Tests for the ignorable and for the MCAR mechanisms are proposed. These tests do not require grouping the individuals by their missing pattern, making them easier to apply in practice. They are based on the estimates of the conditional probabilities of emitting a missing data given the latent state of the Markov chain and some observed covariables. When the ignorable mechanism holds, the conditional probabilities of emitting a missing value are the same for a given value of the observed variables. On the contrary, when the MCAR mechanism holds, these probabilities are all the same. A practical implementation of these tests based on simulations is proposed, along with a presentation of their performances. A real example from piglet farming illustrates their use.

The impact of correlated exposures and missing data on multiple informant models used to identify critical exposure windows.

There has been heightened interest in identifying critical windows of exposure for adverse health outcomes; that is, time points during which exposures have the greatest impact on a person's health. Multiple informant models implemented using generalized estimating equations (MIM GEEs) have been applied to address this research question because they enable statistical comparisons of differences in associations across exposure windows. As interest rises in using MIMs, the feasibility and appropriateness of their application under settings of correlated exposures and partially missing exposure measurements requires further examination. We evaluated the impact of correlation between exposure measurements and missing exposure data on the power and differences in association estimated by the MIM GEE and an inverse probability weighted extension to account for informatively missing exposures. We assessed these operating characteristics under a variety of correlation structures, sample sizes, and missing data mechanisms considering various exposure-outcome scenarios. We showed that applying MIM GEEs maintains higher power when there is a single critical window of exposure and exposure measures are not highly correlated, but may result in low power and bias under other settings. We applied these methods to a study of pregnant women living with HIV to explore differences in association between trimester-specific viral load and infant neurodevelopment.

Diagnosing and Handling Common Violations of Missing at Random

Ignorable likelihood (IL) approaches are often used to handle missing data when estimating a multivariate model, such as a structural equation model. In this case, the likelihood is based on all available data, and no model is specified for the missing data mechanism. Inference proceeds via maximum likelihood or Bayesian methods, including multiple imputation without auxiliary variables. Such IL approaches are valid under a missing at random (MAR) assumption. Rabe-Hesketh and Skrondal (Ignoring non-ignorable missingness. Presidential Address at the International Meeting of the Psychometric Society, Beijing, China, 2015; Psychometrika, 2023) consider a violation of MAR where a variable A can affect missingness of another variable B also when A is not observed. They show that this case can be handled by discarding more data before proceeding with IL approaches. This data-deletion approach is similar to the sequential estimation of Mohan et al. (in: Advances in neural information processing systems, 2013) based on their ordered factorization theorem but is preferable for parametric models. Which kind of data-deletion or ordered factorization to employ depends on the nature of the MAR violation. In this article, we therefore propose two diagnostic tests, a likelihood-ratio test for a heteroscedastic regression model and a kernel conditional independence test. We also develop a test-based estimator that first uses diagnostic tests to determine which MAR violation appears to be present and then proceeds with the corresponding data-deletion estimator. Simulations show that the test-based estimator outperforms IL when the missing data problem is severe and performs similarly otherwise.

İkili Sınıflandırmada Eksik Gözlem Mekanizmalarının Derin Öğrenmeye Etkisi: Bir Benzetim Çalışması

Objective: Investigating the effects of missing data and the methods to overcome problems in statistical models caused by missingness is a significant research topic due to the complex nature of the data, which includes missing observations. The different statistical approaches used in the case of the missing data are complete case analysis and missing data imputation. It is necessary to evaluate missing data mechanisms and patterns to handle missing data issues. However, understanding the missing data mechanism is not easy in relatively large data sets. Recently, deep learning algorithms have been widely used for classification, regression, or clustering tasks in large data sets due to computational advances. The objective of this study is to present the effect of missing data mechanisms on the performance of the deep learning algorithm for binary classification problems. Material and Method: To achieve the aim of this study, an extensive simulation study was conducted using Virtual Machine on Microsoft Azure by considering the missing proportion, the correlation structure, and the mechanism of the missing in the large data set. For different missing data mechanisms, the performance of deep learning with list-wise deletion and imputation compared to the original data set was investigated. Results: It is observed that while the proportion and the mechanism of the missing affect slightly the performance of the deep learning, the correlation level of data affects relatively. Conclusion: Although slight differences were obtained from the area under the curve values, deep learning algorithms can overcome the problem caused by missingness in large data sets.