Expectation Maximization Imputation Research Articles

Predicting insurance claims through a variety of data mining techniques: facing lots of missing values and moderate class-imbalanced levels

Abstract This paper copes with a real-world classification problem related to the management of claims received in an insurance company. The way to obtain the classifier is not easy due to the high amount of missing values as well as the inherent imbalanced scenario within class labels. Once the data partition has been done, the training set is submitted to an intensive double grid search in order to obtain the most promising type of missing value imputation approach and then a step ahead is done using the best method and it starts the next round of data mining strategies which now falls into data rebalancing umbrella. Again, a grid search from an undersampling and oversampling family with different settings is done taking into account only seen data. The training data obtained after the first grid search are now submitted to the second step according the second grid search in order to get the ready training set for the further classifier training. The main objective of the work is to find the best combination of data mining techniques that suits the data set with a pipeline containing two types of data preparation methods coming from different families. As an outcome, first the problem of the presence of missing values has been addressed and then the data rebalancing techniques has been applied. The study focuses on obtaining classifiers based on Bayesian and lazy approaches as well as decision trees, evaluated on metrics such as the area under the ROC curve (AUC), Cohen’s kappa, Accuracy and the F-measure, among others. The imputation by the mean the mode is preferable to the Expectation Maximization Imputation in the scenario faced in this paper taking into account that the amount of missing values is higher than a forty percent for many features.

Machine learning investigation of high-k metal gate processes for dynamic random access memory peripheral transistor

Dynamic random access memory (DRAM) plays a crucial role as a memory device in modern computing, and the high-k/metal gate (HKMG) process is essential for enhancing DRAM’s power efficiency and performance. However, integration of the HKMG process into the existing DRAM technology presents complex and time-consuming challenges. This research uses machine learning analysis to investigate the relationships among the process parameters and electrical properties of HKMG in DRAM. The expectation–maximization imputation was utilized to fill in the missing data, and the Shapley additive explanations analysis was employed for the regression models to predict the electrical properties of HKMG. The impact of the process parameters on the electrical properties is quantified, and the important features that affect the performance of the HKMG transistor are characterized by using the explainable AI algorithm.

BALANCED BAGGING WITH EXPECTATION MAXIMIZATION IMPUTATION IN BANKRUPTCY PREDICTION – APPLICATION ON ROMANIAN COMPANIES

Bankruptcy prediction models are widely used by lending institutions, policy makers or investors. Despite the large volume of international research, limited studies have addressed the particularities of Romanian companies. Balanced Bagging is an Ensemble Method that uses a voting mechanism for a classification task. Expectation Maximization Imputation helps replacing the missing data. In this study we report a promising accuracy performance of 90.03% for the model of Balanced Bagging with Expectation Maximization Imputation on a dataset of more than 20,000 Romanian companies.

Identification of a suitable technique for imputation of incomplete genotyping by sequencing (GBS) data

In the field of DNA sequencing, Genotype by sequencing is to discover SNPs in order to perform Genotyping studies. A most commonly occurring problem in GBS is the presence of missing observations. Quite often, the standard statistical models may not handle such missing data situation also known as incomplete data situations. An alternative to deal with incomplete data situation is to impute missing data for further downstream analysis. Hence a study is conducted with the objectives (i) to impute missing GBS data by various imputation techniques, based on both supervised and unsupervised learning algorithms at different levels of missingness, (ii) to identify suitable imputation technique to deal with incomplete GBS data situation. Based on correlation coefficient and mean squared prediction error (MSPE) between imputed value and true response, the accuracy of imputation technique was assessed. Different imputation techniques, viz., Mean Allele Frequency Imputation (MNI), Singular Value Decomposition Imputation (SVDI), k-Nearest Neighbour Imputation (kNNI), locally weighted linear regression imputation (LWI), Expectation Maximization Imputation (EMI) and Random Forest Imputation (RFI) were applied on incomplete GBS data of mice, a model animal organism, to assess their performance. The results revealed that RFI was found to be most accurate imputation technique. Besides, the performance of RFI in terms correlation coefficient at 5%, 10%, 15% and 20% missing data situation was observed to be 0.778, 0.765, 0.750 and 0.735 respectively. A Similar trend was also observed for RFI in terms of mean square prediction errors. Thus, it is suggested to use RFI technique to deal with incomplete GBS data situation and prior to the application of genomic selection models for breeding value estimation.

An adapted vector autoregressive expectation maximization imputation algorithm for climate data networks

AbstractMissingness in historical climate data networks is a pervasive phenomenon due to the conditions under which these measurements are made. Accurate estimation of these data is a critical issue as projections of future climate depend on a reliable historical climate record. After all, how can the impact of climate change be reliably forecasted when a large proportion of historical climate records are permeated with missing data? We propose an iterative multivariate infilling algorithm and explore its effectiveness on three United States temperature networks of varying densities (number of stations per unit area). Although other multivariate data are not explored here, the application of this infilling procedure is not restricted to climate networks exclusively. We also explore its utility as a function of the type of missing data (i.e., whether or not there is a mechanism or pattern behind the missing observations themselves) and the proportion of missing data within the network. As one may expect, we observe a slightly smaller root mean square error (RMSE) for temperature networks with more stations and less missingness. Somewhat surprisingly, the RMSE tends to be lower for data which is missing at random (there is some mechanism behind the missing data) rather than for data that is missing completely at random (no mechanism behind the missing data). Not surprisingly, the RMSE is largest for data which is missing not at random (missing data are directly related to the values of the observations themselves). Perhaps the most surprising result observed in these networks is that the inclusion of lagged temperature data does not necessarily improve the accuracy of data imputation. The typical RMSE for monthly minimum temperature networks varies from around 1.5 to 2.5°C.This article is categorized under: Data: Types and Structure > Massive Data Data: Types and Structure > Image and Spatial Data Data: Types and Structure > Time Series, Stochastic Processes, and Functional Data Applications of Computational Statistics > Computational Climate Change and Numerical Weather Forecasting

Missing Data Imputation in High Dimensional Data Set using Local Similarity

Data quality is an important aspect for any data mining and statistical tasks. Presence of missing values in the dataset affects the data quality. Missing values refers to the event did not happen or the value does not exist. Data mining algorithms are not robust towards incomplete data. Imputation of missing values is necessary to improve the data quality for performing data mining and statistical analysis. The existing methods such as Expectation Maximization Imputation (EMI), A Framework for Imputing Missing values Using co appearance, correlation and Similarity analysis (FIMUS) use the whole dataset to impute missing values. In such cases, due to the influence of irrelevant record the accuracy of imputation may be affected. This can be controlled by only considering locally similar records to impute missing values. Local similarity imputation can be done through clustering algorithms such as k-means algorithm. K-means clustering efficiency depends on the number of clusters is to be defined by users. To increase the clustering efficiency, first distinctive value is imputed in place of missing ones and this imputed dataset is given to stacked autoencoder for dimensionality reduction which also improves the efficiency of clustering. Initial number of clusters to k-means algorithm is determined using fast clustering. Due to initial imputation, some irrelevant records may be partitioned to a cluster. When these records are used for imputing missing values, accuracy of imputation decreases. In the proposed algorithm, local similarity imputation algorithm uses only top knearest neighbours within the cluster to impute missing values. The performance of the proposed algorithm is evaluated based on Root-Mean-Squared-Error (RMSE) and Index of Agreement (d2). University of California Irvine datasets has been used for analyzing the performance of the proposed algorithm.

Study on missing data imputation and modeling for the leaching process

The leaching process is an important component in hydrometallurgy. A predictive model of the leaching rate lays the foundation for soft measurement and process optimization, and data collection is the key in such a modeling effort. However, because of the complexity and harshness of leaching process, data can only be collected sparsely, which results in data deficiency in the modeling process. Therefore, data imputation before modeling seems to be extremely significant. In this paper, expectation maximization imputation based on the Gaussian mixture model (GMM-EM) and multiple imputation (MI) are respectively applied to perform missing data imputation for leaching process under different data loss rates and data loss patterns, and then the imputation performances are evaluated. Simulation experiment results have shown that GMM-EM and MI both have advantages with regard to data imputation. Therefore, MI based on GMM (GMM-MI), which combines the advantages of GMM and MI, is proposed in this paper. The effectiveness of GMM-MI is verified by a series of simulations.

Missing data imputation on the 5-year survival prediction of breast cancer patients with unknown discrete values

Breast cancer is the most frequently diagnosed cancer in women. Using historical patient information stored in clinical datasets, data mining and machine learning approaches can be applied to predict the survival of breast cancer patients. A common drawback is the absence of information, i.e., missing data, in certain clinical trials. However, most standard prediction methods are not able to handle incomplete samples and, then, missing data imputation is a widely applied approach for solving this inconvenience. Therefore, and taking into account the characteristics of each breast cancer dataset, it is required to perform a detailed analysis to determine the most appropriate imputation and prediction methods in each clinical environment. This research work analyzes a real breast cancer dataset from Institute Portuguese of Oncology of Porto with a high percentage of unknown categorical information (most clinical data of the patients are incomplete), which is a challenge in terms of complexity. Four scenarios are evaluated: (I) 5-year survival prediction without imputation and 5-year survival prediction from cleaned dataset with (II) Mode imputation, (III) Expectation-Maximization imputation and (IV) K-Nearest Neighbors imputation. Prediction models for breast cancer survivability are constructed using four different methods: K-Nearest Neighbors, Classification Trees, Logistic Regression and Support Vector Machines. Experiments are performed in a nested ten-fold cross-validation procedure and, according to the obtained results, the best results are provided by the K-Nearest Neighbors algorithm: more than 81% of accuracy and more than 0.78 of area under the Receiver Operator Characteristic curve, which constitutes very good results in this complex scenario.

Imputation of Unordered Markers and the Impact on Genomic Selection Accuracy

Genomic selection, a breeding method that promises to accelerate rates of genetic gain, requires dense, genome-wide marker data. Genotyping-by-sequencing can generate a large number of de novo markers. However, without a reference genome, these markers are unordered and typically have a large proportion of missing data. Because marker imputation algorithms were developed for species with a reference genome, algorithms suited for unordered markers have not been rigorously evaluated. Using four empirical datasets, we evaluate and characterize four such imputation methods, referred to as k-nearest neighbors, singular value decomposition, random forest regression, and expectation maximization imputation, in terms of their imputation accuracies and the factors affecting accuracy. The effect of imputation method on the genomic selection accuracy is assessed in comparison with mean imputation. The effect of excluding markers with a large proportion of missing data on the genomic selection accuracy is also examined. Our results show that imputation of unordered markers can be accurate, especially when linkage disequilibrium between markers is high and genotyped individuals are related. Of the methods evaluated, random forest regression imputation produced superior accuracy. In comparison with mean imputation, all four imputation methods we evaluated led to greater genomic selection accuracies when the level of missing data was high. Including rather than excluding markers with a large proportion of missing data nearly always led to greater GS accuracies. We conclude that high levels of missing data in dense marker sets is not a major obstacle for genomic selection, even when marker order is not known.

Effects of Item Nonresponses on the Performance of Travel Mode Choice Models for Developing Cities: A Comparative Analysis

To enhance the quality of transportation planning and policy making, it is necessary to properly deal with the nonresponse issues in transport surveys. However, such nonresponse issues especially in developing countries have been ignored in literature. This paper first statistically identifies the missing patterns of item nonresponse (INR) in person trip survey data collected in developing cities and then analyzes the effects of INR on the performance of travel mode choice model (an aggregated multinomial logit model) based on expectation-maximization (EM) imputation method. As a case study, three developing cities representing three levels of INR are analyzed as follow. Firstly, the statistically significant social-economic attributes of trip makers and trip-context factors are identified with respect to INR in the missing pattern analysis by using Chi-Square test method. Secondly, EM imputation based on missing pattern analysis is applied to deal with missing data to obtain the unbiased data set as a benchmark. Thirdly, the null hypothesis that the model parameters estimated with and without imputation are equal is statistically tested using independent-sample T tests and further the internal validity performed in terms of R-squared coefficients is used to identify the discrepancy of model predictions between with and without imputations. Finally, one critical indicator – value of travel time (VOTT) is evaluated considering the effects of missing data. The results confirm that the respondents and non-respondents are quite different in terms of the social-economic background in the developing cities and further show that not only the missing rates but also the missing patterns greatly affect the performance of mode choice model in terms of model parameters and the prediction ability. The calculation of VOTT reveals that the VOTT affected by INR tends to be overestimated.

Is there a role for expectation maximization imputation in addressing missing data in research using WOMAC questionnaire? Comparison to the standard mean approach and a tutorial

BackgroundStandard mean imputation for missing values in the Western Ontario and Mc Master (WOMAC) Osteoarthritis Index limits the use of collected data and may lead to bias. Probability model-based imputation methods overcome such limitations but were never before applied to the WOMAC. In this study, we compare imputation results for the Expectation Maximization method (EM) and the mean imputation method for WOMAC in a cohort of total hip replacement patients.MethodsWOMAC data on a consecutive cohort of 2062 patients scheduled for surgery were analyzed. Rates of missing values in each of the WOMAC items from this large cohort were used to create missing patterns in the subset of patients with complete data. EM and the WOMAC's method of imputation are then applied to fill the missing values. Summary score statistics for both methods are then described through box-plot and contrasted with the complete case (CC) analysis and the true score (TS). This process is repeated using a smaller sample size of 200 randomly drawn patients with higher missing rate (5 times the rates of missing values observed in the 2062 patients capped at 45%).ResultsRate of missing values per item ranged from 2.9% to 14.5% and 1339 patients had complete data. Probability model-based EM imputed a score for all subjects while WOMAC's imputation method did not. Mean subscale scores were very similar for both imputation methods and were similar to the true score; however, the EM method results were more consistent with the TS after simulation. This difference became more pronounced as the number of items in a subscale increased and the sample size decreased.ConclusionsThe EM method provides a better alternative to the WOMAC imputation method. The EM method is more accurate and imputes data to create a complete data set. These features are very valuable for patient-reported outcomes research in which resources are limited and the WOMAC score is used in a multivariate analysis.

The Amyotrophic Lateral Sclerosis Assessment Questionnaire (ALSAQ‐40): Evidence for a method of imputing missing data

The Amyotrophic Lateral Sclerosis Assessment Questionnaire (ALSAQ-40) is the most widely validated measure of health status for use with patients diagnosed with motor neuron disease/amyotrophic lateral sclerosis (MND/ALS). The questionnaire was designed to be used in studies evaluating treatment regimes where missing data may cause problems with data analyses. The purpose of this paper is to evaluate an algorithm for the imputation of missing dimension scores on the ALSAQ-40. We used a postal survey of patients diagnosed with MND/ALS. One thousand, nine hundred and seventy-nine patients were surveyed and 1093 (55.2%) questionnaires returned. Of these, complete ALSAQ-40 data was available in 854 (85.8%) cases. Data were deleted from this complete dataset, and in a randomly selected subset of 100 cases, and then imputed using the Expectation Maximization (EM) algorithm: results were then compared to the dataset prior to data deletion. Descriptive statistics, mean scores and spread of scores were almost identical between original and imputed datasets. Furthermore, the two datasets were highly correlated (intra-class correlation coefficient = 0.95 or greater), and mean differences were small (+/-1.00). We concluded that EM imputation for the ALSAQ-40 provides data that closely mirrors the original when this has been deliberately removed. Consequently, EM is likely to be appropriate for studies using the ALSAQ-40 that contain missing data points.

Analysis of the imputed female urinary incontinence data for the evaluation of expert system parameters

We evaluated parameters for an expert system which will be designed to aid the differential diagnosis of female urinary incontinence by using knowledge discovered from data. To allow the statistical analysis, we applied means, regression and Expectation–Maximization (EM) imputation methods to fill in missing values. In addition, complete-case analysis was performed. Logistic regression results from the imputed data were reasonable. The significant parameters were mostly those that are important in the diagnostic work-up. Moreover, directions of relations between the parameters and the stress, mixed and sensory urge diagnoses were as expected. Analysis with the complete reduced data set gave clearly insufficient results. Imputed values had a moderate agreement, but odds ratios and classification accuracies of logistic regression equations were similar. Results suggest that with these data, simpler methods may be used to allow multivariate analysis and knowledge discovery, when better methods, such as EM imputation, are unavailable. Cluster analysis detected clusters corresponding to the small normal class, but was unable to clearly separate the larger incontinence classes.