Missing Not At Random Research Articles

Nonresponse Bias Analysis in Longitudinal Studies: A Comparative Review with an Application to the Early Childhood Longitudinal Study

SummaryLongitudinal studies are subject to nonresponse when individuals fail to provide data for entire waves or particular questions of the survey. We compare approaches to nonresponse bias analysis (NRBA) in longitudinal studies and illustrate them on the Early Childhood Longitudinal Study, Kindergarten Class of 2010–2011 (ECLS‐K:2011). Wave nonresponse with attrition often yields a monotone missingness pattern, and the missingness mechanism can be missing at random (MAR) or missing not at random (MNAR). We discuss weighting, multiple imputation (MI), incomplete data modelling and Bayesian approaches to NRBA for monotone patterns. Weighting adjustments can be effective when the constructed weights are correlated with the survey outcome of interest. MI allows for variables with missing values to be included in the imputation model, yielding potentially less biased and more efficient estimates. We add offsets in the MAR results to provide sensitivity analyses to assess MNAR deviations. We conduct NRBA for descriptive summaries and analytic model estimates in the ECLS‐K:2011 application. The strength of evidence about our NRBA depends on the strength of the relationship between the fully observed variables and the key survey outcomes, so the key to a successful NRBA is to include strong predictors.

Testing unit root non-stationarity in the presence of missing data in univariate time series of mobile health studies.

The use of digital devices to collect data in mobile health studies introduces a novel application of time series methods, with the constraint of potential data missing at random or missing not at random (MNAR). In time-series analysis, testing for stationarity is an important preliminary step to inform appropriate subsequent analyses. The Dickey-Fuller test evaluates the null hypothesis of unit root non-stationarity, under no missing data. Beyond recommendations under data missing completely at random for complete case analysis or last observation carry forward imputation, researchers have not extended unit root non-stationarity testing to more complex missing data mechanisms. Multiple imputation with chained equations, Kalman smoothing imputation, and linear interpolation have also been used for time-series data, however such methods impose constraints on the autocorrelation structure and impact unit root testing. We propose maximum likelihood estimation and multiple imputation using state space model approaches to adapt the augmented Dickey-Fuller test to a context with missing data. We further develop sensitivity analyses to examine the impact of MNAR data. We evaluate the performance of existing and proposed methods across missing mechanisms in extensive simulations and in their application to a multi-year smartphone study of bipolar patients.

Missing Data Statistics Provide Causal Insights into Data Loss in Diabetes Health Monitoring by Wearable Sensors.

Data loss in wearable sensors is an inevitable problem that leads to misrepresentation during diabetes health monitoring. We systematically investigated missing wearable sensors data to get causal insight into the mechanisms leading to missing data. Two-week-long data from a continuous glucose monitor and a Fitbit activity tracker recording heart rate (HR) and step count in free-living patients with type 2 diabetes mellitus were used. The gap size distribution was fitted with a Planck distribution to test for missing not at random (MNAR) and a difference between distributions was tested with a Chi-squared test. Significant missing data dispersion over time was tested with the Kruskal-Wallis test and Dunn post hoc analysis. Data from 77 subjects resulted in 73 cleaned glucose, 70 HR and 68 step count recordings. The glucose gap sizes followed a Planck distribution. HR and step count gap frequency differed significantly (p < 0.001), and the missing data were therefore MNAR. In glucose, more missing data were found in the night (23:00-01:00), and in step count, more at measurement days 6 and 7 (p < 0.001). In both cases, missing data were caused by insufficient frequency of data synchronization. Our novel approach of investigating missing data statistics revealed the mechanisms for missing data in Fitbit and CGM data.

Missing data imputation using classification and regression trees.

Missing data are common when analyzing real data. One popular solution is to impute missing data so that one complete dataset can be obtained for subsequent data analysis. In the present study, we focus on missing data imputation using classification and regression trees (CART). We consider a new perspective on missing data in a CART imputation problem and realize the perspective through some resampling algorithms. Several existing missing data imputation methods using CART are compared through simulation studies, and we aim to investigate the methods with better imputation accuracy under various conditions. Some systematic findings are demonstrated and presented. These imputation methods are further applied to two real datasets: Hepatitis data and Credit approval data for illustration. The method that performs the best strongly depends on the correlation between variables. For imputing missing ordinal categorical variables, the rpart package with surrogate variables is recommended under correlations larger than 0 with missing completely at random (MCAR) and missing at random (MAR) conditions. Under missing not at random (MNAR), chi-squared test methods and the rpart package with surrogate variables are suggested. For imputing missing quantitative variables, the iterative imputation method is most recommended under moderate correlation conditions.

Deeply Learned Generalized Linear Models with Missing Data

Deep Learning (DL) methods have dramatically increased in popularity in recent years, with significant growth in their application to various supervised learning problems. However, the greater prevalence and complexity of missing data in such datasets present significant challenges for DL methods. Here, we provide a formal treatment of missing data in the context of deeply learned generalized linear models, a supervised DL architecture for regression and classification problems. We propose a new architecture, dlglm, that is one of the first to be able to flexibly account for both ignorable and non-ignorable patterns of missingness in input features and response at training time. We demonstrate through statistical simulation that our method outperforms existing approaches for supervised learning tasks in the presence of missing not at random (MNAR) missingness. We conclude with a case study of the Bank Marketing dataset from the UCI Machine Learning Repository, in which we predict whether clients subscribed to a product based on phone survey data. Supplementary materials for this article are available online.

Multiple imputation of incomplete multilevel data using Heckman selection models.

Missing data is a common problem in medical research, and is commonly addressed using multiple imputation. Although traditional imputation methods allow for valid statistical inference when data are missing at random (MAR), their implementation is problematic when the presence of missingness depends on unobserved variables, that is, the data are missing not at random (MNAR). Unfortunately, this MNAR situation is rather common, in observational studies, registries and other sources of real-world data. While several imputation methods have been proposed for addressing individual studies when data are MNAR, their application and validity in large datasets with multilevel structure remains unclear. We therefore explored the consequence of MNAR data in hierarchical data in-depth, and proposed a novel multilevel imputation method for common missing patterns in clustered datasets. This method is based on the principles of Heckman selection models and adopts a two-stage meta-analysis approach to impute binary and continuous variables that may be outcomes or predictors and that are systematically or sporadically missing. After evaluating the proposed imputation model in simulated scenarios, we illustrate it use in a cross-sectional community survey to estimate the prevalence of malaria parasitemia in children aged 2-10 years in five regions in Uganda.

Evaluating Imputation Methods for rainfall data under high variability in Johor River Basin, Malaysia

Missing values in rainfall records might result in erroneous predictions and inefficient management practices with significant economic, environmental, and social consequences. This is particularly important for rainfall datasets in Peninsular Malaysia (PM) due to the high level of missingness that can affect the inherent pattern in the highly variable time series. In this work, 21 target rainfall stations in the Johor River Basin (JRB) with daily data between 1970 and 2015 were used to examine 19 different multiple imputation methods that were carried out using the Multivariate Imputation by Chained Equations (MICE) package in R. For each station, artificial missing data were added at rates of up to 5%, 10%, 20%, and 30% for different types of missingness, namely, Missing Completely At Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR), leaving the original missing data intact. The imputation quality was evaluated based on several statistical performance metrics, namely mean absolute error (MAE), root mean square error (RMSE), normalized root mean square error (NRMSE), Nash-Sutcliffe efficiency (NSE), modified degree of agreement (MD), coefficient of determination (R2), Kling-Gupta efficiency (KGE), and volumetric efficiency (VE), which were later ranked and aggregated by using the compromise programming index (CPI) to select the best method. The results showed that linear regression predicted values (norm.predict) consistently ranked the highest under all types and levels of missingness. For example, under MAR, MNAR, and MCAR, this method showed the lowest MAE values, ranging between 0.78 and 2.25, 0.93–2.57, and 0.87–2.43, respectively. It also consistently shows higher NSE and R2 values of 0.71–0.92, 0.6–0.92, and 0.66–0.91, and 0.77–0.92, 0.71–0.93, and 0.75–0.92 under MAR, MCAR, and MNAR, respectively. The methods of mean, rf, and cart also appear to be efficient. The incorporation of the compromise programming index (CPI) as a decision-support tool has enabled an objective assessment of the output from the multiple performance metrics for the ranking and selection of the top-performing method. During validation, the Probability Density Function (PDF) demonstrated that even with up to 30% missingness, the shape of the distribution was retained after imputation compared to the actual data. The methodology proposed in this study can help in choosing suitable imputation methods for other tropical rainfall datasets, leading to improved accuracy in rainfall estimation and prediction.

Implication of the 2014 World Health Organization Integrated Management of Childhood Illness Pneumonia Guidelines with and without pulse oximetry use in Malawi: A retrospective cohort study.

Under-5 pneumonia mortality remains high in low-income countries. In 2014 the World Health Organization (WHO) advised that children with chest indrawing pneumonia, but without danger signs or peripheral oxygen saturation (SpO 2) < 90% be treated in the community, rather than hospitalized. In Malawi there is limited pulse oximetry availability. Secondary analysis of 13,413 under-5 pneumonia cases in Malawi. Pneumonia associated case fatality ratios (CFR) were calculated by disease severity under the assumptions of the 2005 and 2014 WHO Integrated Management of Childhood Illness (IMCI) guidelines, with and without pulse oximetry. We investigated if pulse oximetry readings were missing not at random (MNAR). The CFR of patients classified as having non-severe pneumonia per the 2014 IMCI guidelines doubled under the assumption that pulse oximetry was not available (1.5% without pulse oximetry vs 0.7% with pulse oximetry, P<0.001). When 2014 IMCI guidelines were applied with pulse oximetry and a SpO 2 < 90% as the threshold for referral and/or admission, the number of cases meeting hospitalization criteria decreased by 70.3%. Unrecorded pulse oximetry readings were MNAR with an adjusted odds for mortality of 4.9 (3.8, 6.3), similar to that of a SpO 2 < 90%. Although fewer girls were hospitalized, female sex was an independent mortality risk factor. In Malawi, implementation of the 2014 WHO IMCI pneumonia guidelines, without pulse oximetry, will miss high risk cases. Alternatively, implementation of pulse oximetry may result in a large reduction in hospitalization rates without significantly increasing non-severe pneumonia associated CFR if the inability to obtain a pulse oximetry reading is considered a WHO danger sign.

Response Model Selection in Small Area Estimation Under not Missing at Random Nonresponse

Sverchkov and Pfeffermann[ 16 ] consider Small Area Estimation (SAE) under informative probability sampling of areas and within the sampled areas, and not missing at random (NMAR) nonresponse. To account for the nonresponse, the authors assume a given response model, which contains the outcome values as one of the covariates and estimate the corresponding response probabilities by application of the Missing Information Principle, which consists of defining the likelihood as if there was complete response and then integrating out the unobserved outcomes from the likelihood by employing the relationship between the distributions of the observed and the missing data. A key condition for the success of this approach is the ‘correct’ specification of the response model. In this article, we consider the likelihood ratio test and information criteria based on the appropriate likelihood and show how they can be used for the selection of the response model. We illustrate the approach by a small simulation study. AMS Subject Classification: 62D05, 62D10, 62F10

Transformer network for data imputation in electricity demand data

Load forecasting necessitates a significant amount of smart meter data. Several elements in this process, including device malfunctions and signal transmission issues, produce missing data gaps. Missing values in the dataset significantly influence the learning ability of machine learning algorithms, and they must be infilled before proceeding with any statistical analysis. This paper investigates the handling of missing values in demand data, and a new approach is developed for improving the performance of demand analytics, such as energy forecasting. The proposed model uses a transformer neural network to impute the missing values at various rates in the demand profile. Our model uses a k-means algorithm to fill in the missing values with proxy values in the dataset. The model is applied to two case-study residential house located in Cornwall and Fintry, United Kingdom. The developed algorithm is assessed for it potential for infilling missing values for three widely understood missing value scenarios: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). The proposed model's imputed outputs are compared to the original dataset to assess model performance. The performance of the framework is compared with a selection of widely used statistical and machine learning models. The proposed transformer model shows significant improvements over the common linear method in all three scenarios (with 30% missing values), with percentage improvements ranging from approximately 49.71% to 57.52% for Cornwall dataset.

A Machine Learning-Based Multiple Imputation Method for the Health and Aging Brain Study–Health Disparities

The Health and Aging Brain Study–Health Disparities (HABS–HD) project seeks to understand the biological, social, and environmental factors that impact brain aging among diverse communities. A common issue for HABS–HD is missing data. It is impossible to achieve accurate machine learning (ML) if data contain missing values. Therefore, developing a new imputation methodology has become an urgent task for HABS–HD. The three missing data assumptions, (1) missing completely at random (MCAR), (2) missing at random (MAR), and (3) missing not at random (MNAR), necessitate distinct imputation approaches for each mechanism of missingness. Several popular imputation methods, including listwise deletion, min, mean, predictive mean matching (PMM), classification and regression trees (CART), and missForest, may result in biased outcomes and reduced statistical power when applied to downstream analyses such as testing hypotheses related to clinical variables or utilizing machine learning to predict AD or MCI. Moreover, these commonly used imputation techniques can produce unreliable estimates of missing values if they do not account for the missingness mechanisms or if there is an inconsistency between the imputation method and the missing data mechanism in HABS–HD. Therefore, we proposed a three-step workflow to handle missing data in HABS–HD: (1) missing data evaluation, (2) imputation, and (3) imputation evaluation. First, we explored the missingness in HABS–HD. Then, we developed a machine learning-based multiple imputation method (MLMI) for imputing missing values. We built four ML-based imputation models (support vector machine (SVM), random forest (RF), extreme gradient boosting (XGB), and lasso and elastic-net regularized generalized linear model (GLMNET)) and adapted the four ML-based models to multiple imputations using the simple averaging method. Lastly, we evaluated and compared MLMI with other common methods. Our results showed that the three-step workflow worked well for handling missing values in HABS–HD and the ML-based multiple imputation method outperformed other common methods in terms of prediction performance and change in distribution and correlation. The choice of missing handling methodology has a significant impact on the accompanying statistical analyses of HABS–HD. The conceptual three-step workflow and the ML-based multiple imputation method perform well for our Alzheimer’s disease models. They can also be applied to other disease data analyses.

Traffic Status Prediction Based on Multidimensional Feature Matching and 2nd-Order Hidden Markov Model (HMM)

Spatiotemporal data from urban road traffic are pivotal for intelligent transportation systems and urban planning. Nonetheless, missing data in traffic datasets is a common challenge due to equipment failures, communication issues, and monitoring limitations, especially the missing not at random (MNAR) problem. This research introduces an approach to address MNAR-type missing data in traffic status prediction, utilizing a multidimensional feature sequence and a second-order hidden Markov model (2nd-order HMM). First, this approach involves extracting spatiotemporal features for the preset data sections and spatial features for the sections to be predicted based on the traffic spatiotemporal characteristics. Second, using the extracted features, distinctive road traffic features are generated for each section. Furthermore, at specific intervals within the defined time period, nearest distance feature matching is introduced to ascertain the traffic attributes of the road section under prediction. Finally, relying on the matched status results, a 2nd-order HMM is employed to forecast the traffic status for subsequent moments within the defined time period. Experiments were carried out using datasets from Shenzhen City and compared against the hidden Markov models and contrast measure (HMM-C) method to affirm the efficacy of the proposed approach.

Handling missing values in the analysis of between-hospital differences in ordinal and dichotomous outcomes: a simulation study

Missing data are frequently encountered in registries that are used to compare performance across hospitals. The most appropriate method for handling missing data when analysing differences in outcomes between hospitals...

Trial arm outcome variance difference after dropout as an indicator of missing-not-at-random bias in randomized controlled trials.

Randomized controlled trials (RCTs) are vulnerable to bias from missing data. When outcomes are missing not at random (MNAR), estimates from complete case analysis (CCA) and multiple imputation (MI) may be biased. There is no statistical test for distinguishing between outcomes missing at random (MAR) and MNAR. Current strategies rely on comparing dropout proportions and covariate distributions, and using auxiliary information to assess the likelihood of dropout being associated with the outcome. We propose using the observed variance difference across trial arms as a tool for assessing the risk of dropout being MNAR in RCTs with continuous outcomes. In an RCT, at randomization, the distributions of all covariates should be equal in the populations randomized to the intervention and control arms. Under the assumption of homogeneous treatment effects and homoskedastic outcome errors, the variance of the outcome will also be equal in the two populations over the course of follow-up. We show that under MAR dropout, the observed outcome variances, conditional on the variables included in the model, are equal across trial arms, whereas MNAR dropout may result in unequal variances. Consequently, unequal observed conditional trial arm variances are an indicator of MNAR dropout and possible bias of the estimated treatment effect. Heterogeneous treatment effects or heteroskedastic outcome errors are another potential cause of observing different outcome variances. We show that for longitudinal data, we can isolate the effect of MNAR outcome-dependent dropout by considering the variance difference at baseline in the same set of patients who are observed at final follow-up. We illustrate our method in simulation for CCA and MI, and in applications using individual-level data and summarydata.

Leveraging Inverse Probability Weighting Method to Missing Not at Random Problem

In nowadays recommender system field. The selection bias is ubiquitous to most of the data. Most real-world rating data is sparse and missing not at random (MNAR). MNAR data make it difficult to accurately estimate the performance of prediction model as well as learn an optimal prediction model. Investigation on overcoming the MNAR is becoming the mainstream in recommendation field. Plenty of approaches has been proposed to solve this problem. And many of them make the use of the propensity of observing each user-item event. Many of the approaches are suffering from the variance of the propensity. This paper proposes two approaches, summary of inversed propensity (SIP) and clip to make the propensity calculated by logistic regression more effective. And real-world-dataset based experiment will show the performance of these two approaches.

A systematic approach towards missing lab data in electronic health records: A case study in non-small cell lung cancer and multiple myeloma.

Real-world data derived from electronic health records often exhibit high levels of missingness in variables, such as laboratory results, presenting a challenge for statistical analyses. We developed a systematic workflow for gathering evidence of different missingness mechanisms and performing subsequent statistical analyses. We quantify evidence for missing completely at random (MCAR) or missing at random (MAR), mechanisms using Hotelling's multivariate t-test, and random forest classifiers, respectively. We further illustrate how to apply sensitivity analyses using the not at random fully conditional specification procedure to examine changes in parameter estimates under missing not at random (MNAR) mechanisms. In simulation studies, we validated these diagnostics and compared analytic bias under different mechanisms. To demonstrate the application of this workflow, we applied it to two exemplary case studies with an advanced non-small cell lung cancer and a multiple myeloma cohort derived from a real-world oncology database. Here, we found strong evidence against MCAR, and some evidence of MAR, implying that imputation approaches that attempt to predict missing values by fitting a model to observed data may be suitable for use. Sensitivity analyses did not suggest meaningful departures of our analytic results under potential MNAR mechanisms; these results were also in line with results reported in clinical trials.

Improved estimation of hazard function when failure information is missing not at random

ABSTRACT Hazard function plays a crucial role in survival analysis. Its estimation has garnered a lot of attention when the survival time variable suffers from right-censoring. Most of the existing works focus on the cases that failure information is complete or missing at random (MAR). When the censoring information is missing not at random (MNAR), statistical inferences on hazard function are very challenging. In this study, estimation of hazard function is addressed under the MNAR mechanism of the failure information. Three estimators are proposed by employing the techniques of correcting biases and adjusting weighting probabilities. These estimators are validated to be consistent and asymptotically normal. Simulation studies and two real-data analyses are performed to assess the proposed methods.

How to deal with missing data? Multiple imputation by chained equations: recommendations and explanations for clinical practice

The presence of missing data, a constant problem in medical research, has several consequences: systematic loss of power, associated or not with a reduction in the representativeness of the sample analyzed. There are three types of missing data: 1) missing completely at random (MCAR); 2) missing at random (MAR); 3) missing not at random (MNAR). Multiple imputation by chained equations allows for the correct handling of missing data under the MCAR and MAR assumptions. It allows to simulate for each missing data j, a number m of simulated values which seem plausible with regard to the other variables. A random effect is included in this simulation to express the uncertainty. Several data sets are thus created and analyzed individually, in an identical way. Then the estimators of each data set are combined to obtain a global estimator. Multiple imputation increases power, corrects for some biases and has the advantage of being applicable to many types of variables. Complete case analysis should no longer be the norm. The objective of this guide is to help the reader in conducting an analysis with multiple imputed data. We cover the following points: the different types of missing data, the different historical approaches to handling them, and then we detail the multiple imputation method using chained equations. We provide a code example for the mice package of R®.

Implication of the 2014 World Health Organization Integrated Management of Childhood Illness Pneumonia Guidelines with and without pulse oximetry use in Malawi: A retrospective cohort study

Background: Under-5 pneumonia mortality remains high in low-income countries. In 2014 the World Health Organization (WHO) advised that children with chest indrawing pneumonia, but without danger signs or peripheral oxygen saturation (SpO2) &lt; 90% be treated in the community, rather than hospitalized. In Malawi there is limited pulse oximetry availability. Methods: Secondary analysis of 13,413 under-5 pneumonia cases in Malawi. Pneumonia associated case fatality ratios (CFR) were calculated by disease severity under the assumptions of the 2005 and 2014 WHO Integrated Management of Childhood Illness (IMCI) guidelines, with and without pulse oximetry. We investigated if pulse oximetry readings were missing not at random (MNAR). Results: The CFR of patients classified as having non-severe pneumonia per the 2014 IMCI guidelines doubled under the assumption that pulse oximetry was not available (1.5% without pulse oximetry vs 0.7% with pulse oximetry, P&lt;0.001). When 2014 IMCI guidelines were applied with pulse oximetry and a SpO2 &lt; 90% as the threshold for referral and/or admission, the number of cases meeting hospitalization criteria decreased by 70.3%. Unrecorded pulse oximetry readings were MNAR with an adjusted odds for mortality of 4.9 (3.8, 6.3), similar to that of a SpO2 &lt; 90%. Although fewer girls were hospitalized, female sex was an independent mortality risk factor. Conclusions: In Malawi, implementation of the 2014 WHO IMCI pneumonia guidelines, without pulse oximetry, will miss high risk cases. Alternatively, implementation of pulse oximetry may result in a large reduction in hospitalization rates without significantly increasing non-severe pneumonia associated CFR if the inability to obtain a pulse oximetry reading is considered a WHO danger sign.

Data fusion of distance sampling and capture-recapture data

Species distribution models (SDMs) are increasingly used in ecology, biogeography, and wildlife management to learn about the species–habitat relationships and abundance across space and time. Distance sampling (DS) and capture-recapture (CR) are two widely collected data types to learn about species–habitat relationships and abundance; still, they are seldomly used in SDMs due to the lack of spatial coverage. However, data fusion of the two data sources can increase spatial coverage, which can reduce parameter uncertainty and make predictions more accurate, and therefore, can be used with SDM. We developed a model-based approach for data fusion of DS and CR data. Our modeling approach accounts for two common missing data issues: 1) individuals that are missing not at random (MNAR) and 2) partially missing location information. Using a simulation experiment, we evaluated the performance of our modeling approach and compared it to existing approaches that use ad-hoc methods to account for missing data issues. Our results show that our approach provides unbiased parameter estimates with increased efficiency compared to the existing approaches. Finally, we demonstrated our approach using data collected for Grasshopper Sparrows (Ammodramus savannarum) in north-eastern Kansas, USA.