Classification Of Incomplete Data Research Articles

Incomplete data classification via positive approximation based rough subspaces ensemble

Classifying incomplete data using ensemble techniques is a prevalent method for addressing missing values, where multiple classifiers are trained on diverse subsets of features. However, current ensemble-based methods overlook the redundancy within feature subsets, presenting challenges for training robust prediction models, because the redundant features can hinder the learning of the underlying rules in the data. In this paper, we propose a Reduct-Missing Pattern Fusion (RMPF) method to address the aforementioned limitation. It leverages both the advantages of rough set theory and the effectiveness of missing patterns in classifying incomplete data. RMPF employs a heuristic algorithm to generate a set of positive approximation-based attribute reducts. Subsequently, it integrates the missing patterns with these reducts through a fusion strategy to minimize data redundancy. Finally, the optimized subsets are utilized to train a group of base classifiers, and a selective prediction procedure is applied to produce the ensembled prediction results. Experimental results show that our method is superior to the compared state-of-the-art methods in both performance and robustness. Especially, our method obtains significant superiority in the scenarios of data with high missing rates.

Integration of Multikinds Imputation With Covariance Adaptation Based on Evidence Theory.

For incomplete data classification, missing attribute values are often estimated by imputation methods before building classifiers. The estimated attribute values are not actual attribute values. Thus, the distributions of data will be changed after imputing, and this phenomenon often results in degradation of classification performance. Here, we propose a new framework called integration of multikinds imputation with covariance adaptation (MICA) based on evidence theory (ET) to effectively deal with the classification problem with incomplete training data and complete test data. In MICA, we first employ different kinds of imputation methods to obtain multiple imputed training datasets. In general, the distributions of each imputed training dataset and test dataset will be different. A covariance adaptation module (CAM) is then developed to reduce the distribution difference of each imputed training dataset and test dataset. Then, multiple classifiers can be learned on the multiple imputed training datasets, and they are complementary to each other. For a test pattern, we can combine the multiple pieces of soft classification results yielded by these classifiers based on ET to obtain better classification performance. However, the reliabilities/weights of different imputed training datasets are usually different, so the soft classification results cannot be treated equally during fusion. We propose to use covariance difference across datasets and accuracy of imputed training data to estimate the weights. Finally, the soft classification results discounted by the estimated weights are combined by ET to make the final class decision. MICA was compared with a variety of related methods on several datasets, and the experimental results demonstrate that this new method can significantly improve the classification performance.

Category-aware optimal transport for incomplete data classification

Classification with incomplete data is practically challenging since missing data hinders classification. The two-stage strategy of filling missing values first and then classifying complete data degenerates performance due to the separate optimization of classification and imputation. While efforts have been made to classify incomplete data directly without imputation, the benefits of imputation on classification have been sacrificed. To this end, this paper proposes a Category-Aware Optimal Transport Neural Network (CAOT-NN) for incomplete data classification. Specifically, we design a category-aware optimal transport method conditioned on intra-category data distribution, which explicitly employs the category information of each sample and fills in discriminative values for the downstream classification module. Moreover, we reconstruct the observed values during the imputation process when performing classification learning, thus implicitly utilizing the category information of the classifier for missing values estimation. Extensive experiments conducted on 16 real-world and 30 synthetic datasets from the UCI repository demonstrate the superiority of CAOT-NN against existing state-of-the-art methods. Also, the visualization analysis reveals that CAOT-NN enables a tighter intra-category distribution of the imputed data, implying that the missing mechanism may be utilized as a data augmentation strategy to boost accuracy.

An improved feature selection method for classification on incomplete data: Non-negative latent factor-incorporated duplicate MIC

Feature selection is an important data pre-processing procedure in the classification task, whose main purpose is to remove negligible/redundant features to reduce the computational cost, while improving the performance of the subsequent machine learning method. However, most feature selection methods can only work well with the complete data, but show poorly in the presence of the missing information, especially with a high missing rate, due probably to the severe irrelevance and redundancy of features. To address this issue, we proposed a novel feature selection method, namely non-negative latent factor incorporated duplicate maximal information coefficient (NLF-DMIC), which improves the effectiveness of feature selection for the classification of incomplete data. The NLF-DMIC method is fulfilled by the following three steps: (1) select category-friendly features by MIC based on “partial sample strategy” roughly; (2) use the NLF model to make an imputation for the missing data in terms of the filtered features; and (3) select features again by an improved maximal information coefficient (i.e., low-redundancy MIC, LMIC) method on the complete dataset. Finally, experiments on an artificially synthetic dataset and 8 real datasets show that the proposed NLF-DMIC method outperforms some state-of-the-art feature selection methods.

Classification of incomplete data integrating neural networks and evidential reasoning

Classification of incomplete data integrating neural networks and evidential reasoning

MI-MOTE: Multiple imputation-based minority oversampling technique for imbalanced and incomplete data classification

Class imbalance and data incompleteness problems occur simultaneously in many real-world classification datasets, which negatively affects the training of classifiers. Given an imbalanced and incomplete training dataset, the conventional approach is to address these two problems sequentially by handling data incompleteness first and then focusing on class imbalance. In this study, we propose a multiple imputation-based minority oversampling technique, named MI-MOTE, to address imbalanced and incomplete data classification simultaneously. Majority instances are imputed once and minority instances are oversampled using multiple different imputations without directly manipulating any of their observed values. Accordingly, minority instances are diversified with less data distortion compared to the conventional approach. The proposed method is applied in the data preprocessing phase, meaning it can be used with any type of classifier. Experimental results for benchmark datasets with various missing rates demonstrate the effectiveness of the proposed method.

Incomplete data ensemble classification using imputation-revision framework with local spatial neighborhood information

Most existing machine learning techniques require complete data. However, incomplete patterns are common in many real-world scenarios due to the missing values (MVs). Various Missing value imputation (MVI) methods have been proposed to recover the MVs. Each of them has its own advantages in some scenarios. However, on the one hand, few of them consider taking advantages of different MVI methods; on the other hand, how to improve the imputation performance with local information is still an open problem. This paper proposes an imputation-revision framework with local spatial neighborhood information for incomplete data classification. The proposed method endeavors to combine the advantages of several imputation methods. It first obtains several complete datasets which are pre-filled by various MVI methods. Then, it detects the local neighborhood information (LNI) of samples and revises MVs based on the LNI. Finally, ensemble technique is employed to give a final decision. Numerical experiments have verified the superiority of the proposed method in terms of both prediction accuracy and algorithm stability.

Autoencoder-based multi-task learning for imputation and classification of incomplete data

The existence of missing values in real-world datasets increases the difficulty of data analysis. In this paper, we propose an autoencoder (AE)-based multi-task learning (MTL) model and optimize missing values dynamically to classify incomplete datasets having interdependencies among attributes. Specifically, we first design the input structure of hidden neurons in a dynamic way to enhance the imputation performance of AE, and then reorganize the output layer and construct an MTL model to achieve imputation and classification simultaneously. During network training and prediction, missing values are treated as variables and optimized dynamically accompanying with network parameters under the consideration of the incomplete model input. The optimization of missing values promotes the MTL model to match the regression and classification structures implied in incomplete data, thus reducing the impact of the perturbation caused by missing values effectively. The experiments on several datasets validate the effectiveness of the proposed method.

Incomplete data classification with view-based decision tree

Data quality issues may bring serious problems in data analysis. For instance, missing values could decrease the accuracy of the classification. As traditional classification approaches can only be applied to complete data sets, we present a generic classification model for incomplete data where existing classification methods can be effectively incorporated. Firstly, we generate complete views from the incomplete data by choosing proper subsets of attributes based on Information Gain measure. Then we use these selected views to obtain multiple base classifiers. Finally, the base classifies are effectively combined as a final classifier with a decision tree. Extensive experiments results on real data sets demonstrate that the proposed method outperforms existing approaches.

Incomplete data classification—Fisher Discriminant Ratios versus Welch Discriminant Ratios

This study focuses on incomplete data classification with the support of different partial discriminant analyses. When samples contain missing values, discriminant analyses such as Principal Component Analysis and Fisher Discriminant Analysis are inapplicable. Partial discriminant analyses that measure the importance of individual dimensions for incomplete data become necessary. Partial discriminant analyses do not rely on data imputation. The analyses can select and sort dimensions (i.e., predictors) based on discriminability in incomplete data. However, the typical approach Fisher Discriminant Ratios may result in biased estimation due to unequal variance of classes according to statistical literature. This study examines various partial discriminant ratios to discover effective approaches for relieving such a problem by considering different variance in computation. Experiments on an open dataset were carried out during the evaluation. Comparisons included discriminability of Partial Fisher Discriminant Ratios, Partial Welch Discriminant Ratios, and their derivatives in incomplete date classification.

Classification of Incomplete Data Based on Evidence Theory and an Extreme Learning Machine in Wireless Sensor Networks.

In wireless sensor networks, the classification of incomplete data reported by sensor nodes is an open issue because it is difficult to accurately estimate the missing values. In many cases, the misclassification is unacceptable considering that it probably brings catastrophic damages to the data users. In this paper, a novel classification approach of incomplete data is proposed to reduce the misclassification errors. This method uses the regularized extreme learning machine to estimate the potential values of missing data at first, and then it converts the estimations into multiple classification results on the basis of the distance between interval numbers. Finally, an evidential reasoning rule is adopted to fuse these classification results. The final decision is made according to the combined basic belief assignment. The experimental results show that this method has better performance than other traditional classification methods of incomplete data.

Privacy-preserved big data analysis based on asymmetric imputation kernels and multiside similarities

This study presents an efficient approach for incomplete data classification, where the entries of samples are missing or masked due to privacy preservation. To deal with these incomplete data, a new kernel function with asymmetric intrinsic mappings is proposed in this study. Such a new kernel uses three-side similarities for kernel matrix formation. The similarity between a testing instance and a training sample relies not only on their distance but also on the relation between the testing sample and the centroid of the class, where the training sample belongs. This reduces biased estimation compared with typical methods when only one training sample is used for kernel matrix formation. Furthermore, centroid generation does not involve any clustering algorithms. The proposed kernel is capable of performing data imputation by using class-dependent averages. This enhances Fisher Discriminant Ratios and data discriminability. Experiments on two open databases were carried out for evaluating the proposed method. The result indicated that the accuracy of the proposed method was higher than that of the baseline. These findings thereby demonstrated the effectiveness of the proposed idea.

Dynamic Cost-Sensitive Extreme Learning Machine for Classification of Incomplete Data Based on the Deep Imputation Network

Due to its importance in many applications, the incomplete data mining has received increasing attention in recent years, but there has been little study of the cost-sensitive classification on incomplete data. Therefore this paper proposes the dynamic costsensitive extreme learning machine for classification of incomplete data based on the deep imputation network (DCELMIDC). Firstly, we propose an approach for incomplete data imputation based on the deep imputation network model, and offer the cost-sensitive extreme learning machine. Secondly, this paper introduces dynamic misclassification and test cost, and gives the chromosome coding and an evaluation method of the optimal cost. At last, on the basis of the genetic algorithm, the dynamic cost-sensitive extreme learning machine classification algorithm for mining incomplete data is given, which can search the optimal misclassification and test cost in cost spaces. The experiment results show that DCELMIDC is effective and feasible for classification of incomplete data, and can reduce the total cost.

Incomplete data classification with voting based extreme learning machine

Extreme learning machine (ELM) was proposed as a new efficient learning algorithm for single-hidden layer feedforward neural networks (SLFN) in recent years. It is featured by its much faster training speed and better generalization performance over traditional SLFN learning techniques. However, ELM cannot deal directly with incomplete data which widely exists in real-world applications. In this paper, we propose a new algorithm to handle incomplete data with voting based extreme learning machine (V-ELMI). V-ELMI did not rely on any assumptions about missing values. It first obtains a group of data subsets according to the missing values of the training set. Then, it applies mutual information to measure the importance degree of each data subsets. After that, it trains a group of subclassifiers on these data subsets by applying ELM as base learning algorithm. Finally, for a given test sample with missing values, V-ELMI selects the subclassifiers whose input did not require the missing values to predict it. And final prediction is determined by weighted majority voting according to the mean value of the norms of the output weights and the importance degree of each available subclassifier. Experimental results on 15 UCI incomplete datasets and 5 UCI complete datasets have shown that, V-ELMI generally has better performance than the algorithms compared. Moreover, compared with the classification algorithms based on neural network ensemble (NNE), V-ELMI can greatly improve algorithm computational efficiency.

Classification of incomplete data based on belief functions and K-nearest neighbors

It can be quite difficult to correctly and precisely classify the incomplete data with missing values, since the missing information usually causes ambiguities (uncertainty) in the classification result. Belief function theory can well model such uncertain and imprecise information, and a new belief-based method for credal classification of incomplete data (CCI) is proposed using the K nearest neighbors (KNNs) strategy. In CCI, the KNNs of object (incomplete data) are respectively used to estimate the missing values, and one can obtain K versions of edited pattern with estimated values from the KNNs. The K edited patterns are classified by any classical method to get K pieces of classification results with different discounting (weighting) factors depending on the distances between the object and its KNNs, and global fusion of the K classification results represented by the basic belief assignments (bba’s) is used for credal classification of the object. The conflicting beliefs produced in the fusion process can well capture the imprecision degree of classification, and it will be transferred to the selected meta-class defined by the disjunction of several classes (i.e. the set of several classes) according to the current context. Thus, the incomplete data that is hard to correctly classify because of the missing values will be reasonably committed to proper meta-class, which is able to characterize the imprecision of classification and reduce the errors as well. Three experiments are given to illustrate the potential and interest of CCI approach.

Land-cover classification of partly missing data using support vector machines

Land-cover classification based on multi-temporal satellite images for scenarios where parts of the data are missing due to, for example, clouds, snow or sensor failure has received little attention in the remote-sensing literature. The goal of this article is to introduce support vector machine (SVM) methods capable of handling missing data in land-cover classification. The novelty of this article consists of combining the powerful SVM regularization framework with a recent statistical theory of missing data, resulting in a new method where an SVM is trained for each missing data pattern, and a given incomplete test vector is classified by selecting the corresponding SVM model. The SVM classifiers are evaluated on Landsat Enhanced Thematic Mapper Plus (ETM + ) images covering a scene of Norwegian mountain vegetation. The results show that the proposed SVM-based classifier improves the classification accuracy by 5–10% compared with single image classification. The proposed SVM classifier also outperforms recent non-parametric k-nearest neighbours (k-NN) and Parzen window density-based classifiers for incomplete data by about 3%. Moreover, since the resulting SVM classifier may easily be implemented using existing SVM libraries, we consider the new method to be an attractive choice for classification of incomplete data in remote sensing.

On Classification with Incomplete Data

We address the incomplete-data problem in which feature vectors to be classified are missing data (features). A (supervised) logistic regression algorithm for the classification of incomplete data is developed. Single or multiple imputation for the missing data is avoided by performing analytic integration with an estimated conditional density function (conditioned on the observed data). Conditional density functions are estimated using a Gaussian mixture model (GMM), with parameter estimation performed using both Expectation-Maximization (EM) and Variational Bayesian EM (VB-EM). The proposed supervised algorithm is then extended to the semisupervised case by incorporating graph-based regularization. The semisupervised algorithm utilizes all available data-both incomplete and complete, as well as labeled and unlabeled. Experimental results of the proposed classification algorithms are shown.