Imputation Model Research Articles

Association of the cognitive reserve with the age of onset of mild cognitive impairment and dementia in patients with or Alzheimer’s disease. Antioquia, Colombia.

AbstractBackgroundThe cognitive reserve (CR) has been associated with the extension/compensation of the mental functioning of individuals affected by a neurodegenerative disease in the early stages of the disease. Most of the analyses evaluating CR are in high‐income countries, while there is a lack of capturing the variability related to low‐middle‐income environments such as Colombia. Objective: to analyze the relationship between CR and the age of onset of MCI and/or dementia by Alzheimer’s disease (AD) in patients from Antioquia‐Colombia.MethodA retrospective cohort study of 194 individuals. Clinical and socioeconomic characteristics were described using absolute and relative frequencies, followed by a mixed principal component analysis to represent the CR. A Varimax rotation was used for its interpretation, generating Z scores for the CR and the resulting dimensions. CR was categorized according to the median as high or low. This categorization aims to assess differences in the onset time of MCI and dementia by AD for the survival analysis and Cox regression with multiple imputation models (m = 30). We also examined all data to estimate the raw effect and adjusted for age, sex, and area.Resultsfrom 194 patients, 151 informed the age of onset of MCI, and 165 were later diagnosed with dementia by AD. There were differences between 5 and 9 years in the median age of onset of MCI and dementia by AD, with a later onset for patients with high CR. The CR hazard ratios (HR) were 1.05 and 0.97 for MCI and dementia, respectively. The ratio for the total score tended to be null after adjustment with confounding variables. However, its dimensions remained significant. Specifically, the HRs for an academic position, growing environment, and family status increased the speed of onset of the outcomes for MCI: 1.43, 1.06, and 1,72 and dementia: 1.40, 1.12, and 1.68.Conclusionthe onset times of the MCI and dementia were five years later in subjects with higher CR. Thus, a high academic position, a protective growth environment, and a family status that favors social relationships were associated with the reduction in clinical symptoms of AD since the early stages.

Efficient imputation of missing data using the information of local space defined by the geometric one-class classifier

Datasets gathered from actual systems may include missing data owing to unintentional faults, such as the breakdown of equipment as well as intentional reasons such as sampling inspection. Because missing data can result in incorrect and distorted results when analyzed, they should be addressed before the analysis is performed. Imputation of missing data involves replacing missing entries of data with values calculated from observed features, which is a more reasonable alternative than simple methods, including a complete case analysis. Although various imputation methods exist for missing data, most ignore the local space around it, which may be closely related to missing values. Furthermore, the imputation method, which can partially reflect local relationships, is susceptible to overfitting and has parameter tuning issues owing to the lack of a systematic definition of the local space. Thus, we propose a composite fuzzy hyper-rectangle (H-RTGL) imputation (CFHRI) method with the following characteristics: (i) it defines the local space using an H-RTGL-based one-class classifier to thoroughly describe the data of the target class, and (ii) it imputes the missing entries using a fuzzy model comprising imputation models calculated from H-RTGLs. These features enable CFHRI to formulate the local space adjacent to missing data systematically and alleviate the hazards of overfitting into a certain region of the dataset. We validated our method based on numerical experiments conducted using a dataset gathered from an actual system and comparison of the imputation performance of our method with that of other imputation methods. CFHRI showed statistically significant improvement in 5 datasets among 7 datasets used, and around 10% enhanced in terms of Mean Absolute Error (MAE). Moreover, we could achieve 3–5% of increased classification accuracy of imputed dataset, which indicates CFHRI can be a useful pre-processor of dataset whose purpose is classification.

Towards real-world traffic prediction and data imputation: A multi-task pretraining and fine-tuning approach

Real-world traffic prediction is challenging and requires accuracy, efficiency, and generalizability for applications. Most studies used two-step data imputation-prediction procedures or recurrent networks to perform predictions based on imputed data, resulting in error accumulation. Training the same model independently for different missing rates and missing data patterns requires high computational costs. This strategy cannot be integrated with existing methods, causing low generalizability. We propose a multi-task pretraining and fine-tuning (MTPF) approach to address these issues. Specifically, we pretrain an encoder and decoders with different missing data patterns and missing rates and an adversarial noise to learn the correlations between different scenarios and generate robust hidden representations. The encoder and decoder perform fine-tuning for specific missing data scenarios after pretraining. Multiple parallel tasks, including prediction, data imputation, contrastive learning, and adaptive graph learning, are used for pretraining to enable the model to learn the hidden representation for prediction and data imputation during fine-tuning. The MTPF is compared with nine baseline models for prediction and five baseline models for data imputation using two real-world datasets. The experimental results show that MTPF outperforms many state-of-the-art baseline models, has a shorter training time, and is a model-agnostic off-the-shelf plug-in that improves baseline model performance.

An effective variational auto-encoder-based model for traffic flow imputation

An effective variational auto-encoder-based model for traffic flow imputation

Subgraph extraction and graph representation learning for single cell Hi-C imputation and clustering.

Single-cell Hi-C (scHi-C) technology enables the investigation of 3D chromatin structure variability across individual cells. However, the analysis of scHi-C data is challenged by a large number of missing values. Here, we present a scHi-C data imputation model HiC-SGL, based on Subgraph extraction and graph representation learning. HiC-SGL can also learn informative low-dimensional embeddings of cells. We demonstrate that our method surpasses existing methods in terms of imputation accuracy and clustering performance by various metrics.

A time series continuous missing values imputation method based on generative adversarial networks

Generative adversarial networks (GANs) have been widely utilized in time series analysis and modeling, wherein generators and discriminators interact to generate realistic data. However, when addressing the challenge of imputing continuous missing values in time series, the generator struggles to learn meaningful features due to the loss of local information, and the discriminator’s loss function exhibits significant deviations in determining the probability, making it difficult to effectively update model parameters during training. In response to these issues, this study presents a novel imputation model called cue Wasserstein generative adversarial network with gradient penalty (CWGAIN-GP). CWGAIN-GP incorporates the contextual cue information matrixs into the generator, enabling obtain and capture the potential data evolution features that hidden beyond the missing positions. This approach constrains the generator’s generation results to be closer to the true values probability distribution. Meanwhile, it optimizes the generator and discriminator neural network structure and loss function computation strategy of the original GAN model, which not only improves the accuracy of continuous missing data interpolation, but also improves the training stability of the model. Finally, We used three imputation accuracies on two real-world datasets, including experiments on imputation accuracy for eight comparison models, ablation study, and experiments on the effect of consecutive missing lengths. The experimental results show that CWGAIN-GP achieves impressive performance in imputing continuous missing time series.

On generating plausible values for multilevel modelling with large-scale-assessment data.

Large-scale assessments (LSAs) routinely employ latent regressions to generate plausible values (PVs) for unbiased estimation of the relationship between examinees' background variables and performance. To handle the clustering effect common in LSA data, multilevel modelling is a popular choice. However, most LSAs use single-level conditioning methods, resulting in a mismatch between the imputation model and the multilevel analytic model. While some LSAs have implemented special techniques in single-level latent regressions to support random-intercept modelling, these techniques are not expected to support random-slope models. To address this gap, this study proposed two new single-level methods to support random-slope estimation. The existing and proposed methods were compared to the theoretically unbiased multilevel latent regression method in terms of their ability to support multilevel models. The findings indicate that the two existing single-level methods can support random-intercept-only models. The multilevel latent regression method provided mostly adequate estimates but was limited by computational burden and did not have the best performance across all conditions. One of our proposed single-level methods presented an efficient alternative to multilevel latent regression and was able to recover acceptable estimates for all parameters. We provide recommendations for situations where each method can be applied, with some caveats.

Multiple imputation of missing data in multilevel ecological momentary assessments: an example using smoking cessation study data.

Advances in digital technology have greatly increased the ease of collecting intensive longitudinal data (ILD) such as ecological momentary assessments (EMAs) in studies of behavior changes. Such data are typically multilevel (e.g., with repeated measures nested within individuals), and are inevitably characterized by some degrees of missingness. Previous studies have validated the utility of multiple imputation as a way to handle missing observations in ILD when the imputation model is properly specified to reflect time dependencies. In this study, we illustrate the importance of proper accommodation of multilevel ILD structures in performing multiple imputations, and compare the performance of a multilevel multiple imputation (multilevel MI) approach relative to other approaches that do not account for such structures in a Monte Carlo simulation study. Empirical EMA data from a tobacco cessation study are used to demonstrate the utility of the multilevel MI approach, and the implications of separating participant- and study-initiated EMAs in evaluating individuals' affective dynamics and urge.

Missing Data Imputation with High-Dimensional Data

Imputation of missing data in high-dimensional datasets with more variables P than samples N, P ≫ N , is hampered by the data dimensionality. For multivariate imputation, the covariance matrix is ill conditioned and cannot be properly estimated. For fully conditional imputation, the regression models for imputation cannot include all the variables. Thus, the high dimension requires special imputation approaches. In this article, we provide an overview and realistic comparisons of imputation approaches for high-dimensional data when applied to a linear mixed modeling (LMM) framework. We examine approaches from three different classes using simulation studies: multiple imputation with penalized regression, multiple imputation with recursive partitioning and predictive mean matching; and multiple imputation with Principal Component Analysis (PCA). We illustrate the methods on a real case study where a multivariate outcome (i.e., an extracted set of correlated biomarkers from human urine samples) was collected and monitored over time and we discuss the proposed methods with more standard imputation techniques that could be applied by ignoring either the multivariate or the longitudinal dimension. Our simulations demonstrate the superiority of the recursive partitioning and predictive mean matching algorithm over the other methods in terms of bias, mean squared error and coverage of the LMM parameter estimates when compared to those obtained from a data analysis without missingness, although it comes at the expense of high computational costs. It is worthwhile reconsidering much faster methodologies like the one relying on PCA.

Kamel: A Scalable BERT-Based System for Trajectory Imputation

Numerous important applications rely on detailed trajectory data. Yet, unfortunately, trajectory datasets are typically sparse with large spatial and temporal gaps between each two points, which is a major hurdle for their accuracy. This paper presents Kamel; a scalable trajectory imputation system that inserts additional realistic trajectory points, boosting the accuracy of trajectory applications. Kamel maps the trajectory imputation problem to finding the missing word problem; a classical problem in the natural language processing (NLP) community. This allows employing the widely used BERT model for trajectory imputation. However, BERT, as is, does not lend itself to the special characteristics of trajectories. Hence, Kamel starts from BERT, but then adds spatial-awareness to its operations, adjusts trajectory data to be closer to the nature of language data, and adds multipoint imputation ability to it; all encapsulated in one system. Experimental results based on real datasets show that Kamel significantly outperforms its competitors and is applicable to city-scale trajectories, large gaps, and tight accuracy thresholds.

ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection

Anomaly detection in multivariate time series data is of paramount importance for large-scale systems. However, accurately detecting anomalies in such data poses significant challenges due to the need for precise data modeling capability. Existing forecasting and reconstruction-based methods struggle to address these challenges effectively. To overcome these limitations, we propose a novel anomaly detection framework named ImDiffusion, which combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. The imputation-based approach employed by ImDiffusion leverages the information from neighboring values in the time series, enabling precise modeling of temporal and inter-correlated dependencies, reducing uncertainty in the data, thereby enhancing the robustness of the anomaly detection process. ImDiffusion further leverages diffusion models as time series imputers to accurately capture complex dependencies. We leverage the step-by-step denoised outputs generated during the inference process to serve as valuable signals for anomaly prediction, resulting in improved accuracy and robustness of the detection process. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets. The results demonstrate that our proposed framework significantly outperforms state-of-the-art approaches in terms of detection accuracy and timeliness. ImDiffusion is further integrated into the real production system in Microsoft and observes a remarkable 11.4% increase in detection F1 score compared to the legacy approach. To the best of our knowledge, ImDiffusion represents a pioneering approach that combines imputation-based techniques with time series anomaly detection, while introducing the novel use of diffusion models to the field.

Making Data Count: Estimating a Poverty Trend for Nigeria between 2009 and 2019

Abstract Monitoring poverty reduction requires frequent microdata on household welfare that can be compared over time. Such data are unavailable in many countries, given limited statistical capacity, shocks that prevent data collection, and regular improvements to survey methodology. This paper demonstrates how jointly deploying backcasting and survey-to-survey imputations can help to overcome this in a setting where estimating a poverty trend is badly needed, given the scale of the poverty-reduction challenge, but where survey-to-survey imputations are more likely to succeed and can be directly tested. In Nigeria, the most recent official survey that can be used to construct an imputation model was collected through the same methodology and in the same year as the target survey. This data landscape could arise in other settings where the methodology for smaller, interstitial surveys is updated more quickly than for larger, official consumption surveys. Naively comparing Nigeria's last two official consumption surveys would suggest that the poverty rate fell by 17 percentage points between 2009 and 2019. Yet the methods presented in this paper both suggest a much smaller reduction in poverty of between 3 and 7 percentage points, echoing Nigeria's performance on nonmonetary welfare indicators over the same period. The paper therefore provides guidance on when and how backcasting and survey-to-survey imputation techniques can be most valuable for monitoring poverty reduction.

Imputation of missing daily rainfall data; A comparison between artificial intelligence and statistical techniques

Handling missing values is a critical component of the data processing in hydrological modeling. The key objective of this research is to assess statistical techniques (STs) and artificial intelligence-based techniques (AITs) for imputing missing daily rainfall values and recommend a methodology applicable to the mountainous terrain of northern Thailand. In this study, 30 years of daily rainfall data was collected from 20 rainfall stations in northern Thailand and randomly 25–35 % of data was deleted from four target stations based on Spearman correlation coefficient between the target and neighboring stations. Imputation models were developed on training and testing datasets and statistically evaluated by mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2), and correlation coefficient (r). This study used STs, including arithmetic averaging (AA), multiple linear regression (MLR), normal-ratio (NR), nonlinear iterative partial least squares (NIPALS) algorithm, and linear interpolation was used.•STs results were compared with AITs, including long-short-term-memory recurrent neural network (LSTM-RNN), M5 model tree (M5-MT), multilayer perceptron neural networks (MLPNN), support vector regression with polynomial and radial basis function SVR-poly and SVR-RBF.•The findings revealed that MLR imputation model achieved an average MAE of 0.98, RMSE of 4.52, and R2 was about 79.6 % at all target stations. On the other hand, for the M5-MT model, the average MAE was 0.91, RMSE was about 4.52, and R2 was around 79.8 % compared to other STs and AITs. M5-MT was most prominent among AITs. Notably, the MLR technique stood out as a recommended approach due to its ability to deliver good estimation results while offering a transparent mechanism and not necessitating prior knowledge for model creation.

Distribution system state estimation with Transformer-Bi-LSTM-based imputation model for missing measurements

Distribution system state estimation with Transformer-Bi-LSTM-based imputation model for missing measurements

Multiple Imputation Through XGBoost

The use of multiple imputation (MI) is becoming increasingly popular for addressing missing data. Although some conventional MI approaches have been well studied and have shown empirical validity, they have limitations when processing large datasets with complex data structures. Their imputation performances usually rely on the proper specification of imputation models, and this requires expert knowledge of the inherent relations among variables. Moreover, these standard approaches tend to be computationally inefficient for medium and large datasets. In this article, we propose a scalable MI framework mixgb, which is based on XGBoost, subsampling, and predictive mean matching. Our approach leverages the power of XGBoost, a fast implementation of gradient boosted trees, to automatically capture interactions and nonlinear relations while achieving high computational efficiency. In addition, we incorporate subsampling and predictive mean matching to reduce bias and to better account for appropriate imputation variability. The proposed framework is implemented in an R package mixgb. Supplementary materials for this article are available online.

A Machine Learning-Based Multiple Imputation Method for the Health and Aging Brain Study–Health Disparities

The Health and Aging Brain Study–Health Disparities (HABS–HD) project seeks to understand the biological, social, and environmental factors that impact brain aging among diverse communities. A common issue for HABS–HD is missing data. It is impossible to achieve accurate machine learning (ML) if data contain missing values. Therefore, developing a new imputation methodology has become an urgent task for HABS–HD. The three missing data assumptions, (1) missing completely at random (MCAR), (2) missing at random (MAR), and (3) missing not at random (MNAR), necessitate distinct imputation approaches for each mechanism of missingness. Several popular imputation methods, including listwise deletion, min, mean, predictive mean matching (PMM), classification and regression trees (CART), and missForest, may result in biased outcomes and reduced statistical power when applied to downstream analyses such as testing hypotheses related to clinical variables or utilizing machine learning to predict AD or MCI. Moreover, these commonly used imputation techniques can produce unreliable estimates of missing values if they do not account for the missingness mechanisms or if there is an inconsistency between the imputation method and the missing data mechanism in HABS–HD. Therefore, we proposed a three-step workflow to handle missing data in HABS–HD: (1) missing data evaluation, (2) imputation, and (3) imputation evaluation. First, we explored the missingness in HABS–HD. Then, we developed a machine learning-based multiple imputation method (MLMI) for imputing missing values. We built four ML-based imputation models (support vector machine (SVM), random forest (RF), extreme gradient boosting (XGB), and lasso and elastic-net regularized generalized linear model (GLMNET)) and adapted the four ML-based models to multiple imputations using the simple averaging method. Lastly, we evaluated and compared MLMI with other common methods. Our results showed that the three-step workflow worked well for handling missing values in HABS–HD and the ML-based multiple imputation method outperformed other common methods in terms of prediction performance and change in distribution and correlation. The choice of missing handling methodology has a significant impact on the accompanying statistical analyses of HABS–HD. The conceptual three-step workflow and the ML-based multiple imputation method perform well for our Alzheimer’s disease models. They can also be applied to other disease data analyses.

Calibrated regression estimation using empirical likelihood under data fusion

Data analysis based on information from different sources, typically known as the data fusion problem, is common in economic and biomedical studies. An interesting question concerns the regression of an outcome variable on certain covariates when combining two distinct datasets. These datasets consist of a primary sample containing the outcome and a subset of the covariates, and a supplemental sample comprising information only on the full set of covariates. Previous methods have proposed doubly robust estimation procedures that employ a single propensity score model for the data fusion process and a single imputation model for the covariates available only in the supplemental dataset. However, it may be questionable to assume that either model is correctly specified due to an unknown data generating process. To address this issue, an empirical likelihood based approach that calibrates multiple propensity scores and imputation models is introduced. The resulting estimator is consistent when any one of the models is correctly specified and is robust against extreme values of the fitted propensity scores. The asymptotic normality property and the estimation efficiency are also discussed. Simulation studies show that the proposed estimator has substantial advantages over existing estimators, and an assembled U.S. household expenditure data example is used for illustration.

Calibrated imputation for multivariate categorical data

Non-response is a major problem for anyone collecting and processing data. A commonly used technique to deal with missing data is imputation, where missing values are estimated and filled in into the dataset. Imputation can become challenging if the variable to be imputed has to comply with a known total. Even more challenging is the case where several variables in the same dataset need to be imputed and, in addition to known totals, logical restrictions between variables have to be satisfied. In our paper, we develop an approach for a broad class of imputation methods for multivariate categorical data such that previously published totals are preserved while logical restrictions on the data are satisfied. The developed approach can be used in combination with any imputation model that estimates imputation probabilities, i.e. the probability that imputation of a certain category for a variable in a certain unit leads to the correct value for this variable and unit.

MIIPW: An R package for Generalized Estimating Equations with missing data integration using a combination of mean score and inverse probability weighted approaches and multiple imputation

The presence of missing values in longitudinal cohort data can lead to inaccuracies in parameter estimation and decreased statistical power. To address this issue, we have developed the MIIPW R package, which incorporates the mean score and inverse probability weighted methods in generalized estimating equations to impute missing scores due to incomplete covariate data. Augmentation of incomplete data in the estimating equation is done through a multiple imputation model. The package employs various methods such as mean score, SIPW, AIPW, miSIPW, and miAIPW to estimate parameters while considering four different covariance structures (AR-1, Exchangeable, Unstructured, and Independent). Additionally, the package uses the QIC for model selection and pays special attention to the calculation of weights within the dataset. The performance of the above mentioned methods has been evaluated through simulation study as well as real data analysis. The MIIPW package is available for download from the comprehensive R archive network and this article provides a practical guide to solve missing data issues. https://CRAN.R-project.org/package=MIIPW.

Attention-based generative adversarial networks for aquaponics environment time series data imputation

Environmental parameter data collected by sensors for monitoring the environment of agricultural facility operations are usually incomplete due to external environmental disturbances and device failures. And the missing of collected data is completely at random. In practice, missing data could create biased estimations and make multivariate time series predictions of environmental parameters difficult, leading to imprecise environmental control. A multivariate time series imputation model based on generative adversarial networks and multi-head attention (ATTN-GAN) is proposed in this work to reducing the negative consequence of missing data. ATTN-GAN can capture the temporal and spatial correlation of time series, and has a good capacity to learn data distribution. In the downstream experiments, we used ATTN-GAN and baseline models for data imputation, and predicted the imputed data, respectively. For the imputation of missing data, over the 20%, 50% and 80% missing rate, ATTN-GAN had the lowest RMSE, 0.1593, 0.2012 and 0.2688 respectively. For water temperature prediction, data processed with ATTN-GAN over MLP, LSTM, DA-RNN prediction methods had the lowest MSE, 0.6816, 0.8375 and 0.3736 respectively. Those results revealed that ATTN-GAN outperformed all baseline models in terms of data imputation accuracy. The data processed by ATTN-GAN is the best for time series prediction.