Imbalanced Data Classification Algorithm Research Articles

Imbalanced data classification algorithm with support vector machine kernel extensions

Learning from the imbalanced data samples so as to achieve accurate classification is an important research content in data mining field. It is very difficult for classification algorithm to achieve a higher accuracy because the uneven distribution of data samples makes some categories have few samples. A imbalanced data classification algorithm of support vector machines (KE-SVM) is proposed in this article, this algorithm achieve the initial classification of data samples by training the maximum margin classification SVM model, and then obtaining a new kernel extension function. based on Chi square test and weight coefficient calculation, through training the samples again by the new vector machine with kernel function to improve the classification accuracy. Through the simulation experiments of real data sets of artificial data set, it shows that the proposed method has higher classification accuracy and faster convergence for the uneven distribution data.

Deep Weighted Extreme Learning Machine

The imbalanced data classification attracts increasing attention in the past years due to the continuous expansion of data available in many areas, such as biomedical engineering, surveillance, and computer vision. Learning from imbalanced data is challenging as most standard algorithms fail to properly represent the inherent complex characteristics of the data distribution. As an emerging technology, the extreme learning machine (ELM) and its variants, including the weighted ELM (WELM) and the boosting weighted ELM (BWELM), have been recently developed for the classification of imbalanced data. However, the WELM suffers the following deficiencies: (i) the sample weight matrix is manually chosen and fixed during the learning phase; (ii) the representation capability, namely the capability to extract features or useful information from the original data, is insufficiently explored due to its shallow structure. The BWELM employs the AdaBoost algorithm to optimize the sample weights. But the representation capability is still restricted by the shallow structure. To alleviate these deficiencies, we propose a novel deep weighted ELM (DWELM) algorithm for imbalanced data classification in this paper. An enhanced stacked multilayer deep representation network trained with the ELM (EH-DrELM) is first proposed to improve the representation capability, and a fast AdaBoost algorithm for imbalanced multiclass data (AdaBoost-ID) is developed to optimize the sample weights. Then, the novel DWELM for the imbalance learning is obtained by combining the above two algorithms. Experimental results on nine imbalanced binary-class datasets, nine imbalanced multiclass datasets, and five large benchmark datasets (three for multiclass and two for binary-class) show that the proposed DWELM achieves a better performance than the WELM and BWELM, as well as several state-of-the-art multilayer network-based learning algorithms.

Feature Learning With a Divergence-Encouraging Autoencoder for Imbalanced Data Classification

Imbalanced data exists commonly in machine learning classification applications. Popular classification algorithms are based on the assumption that data in different classes are roughly equally distributed; however, extremely skewed data, with instances from one class taking up most of the dataset, is not exceptional in practice. Thus, performance of algorithm often degrades significantly when encountering skewed data. Mitigating the problem caused by imbalanced data has been an open challenge for years, and previous researches mostly have proposed solutions from the perspectives of data re-sampling and algorithm improvement. In this paper, focusing on two-class imbalanced data, we have proposed a novel divergence-encouraging autoencoder (DEA) to explicitly learn features from both of the two classes and have designed an imbalanced data classification algorithm based on the proposed autoencoder. By encouraging maximization of divergence loss between different classes in the bottleneck layer, the proposed DEA can learn features for both majority and minority classes simultaneously. The training procedure of the proposed autoencoder is to alternately optimize reconstruction and divergence losses. After obtaining the features, we directly compute the cosine distances between the training and testing features and compare the median of distances between classes to perform classification. Experimental results illustrate that our algorithm outperform ordinary and loss-sensitive CNN models both in terms of performance evaluation metrics and convergence properties. To the best of our knowledge, this is the first paper proposed to solve the imbalanced data classification problem from the perspective of explicitly learning representations of different classes simultaneously. In addition, designing of the proposed DEA is also an innovative work, which could improve the performance of imbalanced data classification without data re-sampling and benefit future researches in the field.

CCR: A combined cleaning and resampling algorithm for imbalanced data classification

Abstract Imbalanced data classification is one of the most widespread challenges in contemporary pattern recognition. Varying levels of imbalance may be observed in most real datasets, affecting the performance of classification algorithms. Particularly, high levels of imbalance make serious difficulties, often requiring the use of specially designed methods. In such cases the most important issue is often to properly detect minority examples, but at the same time the performance on the majority class cannot be neglected. In this paper we describe a novel resampling technique focused on proper detection of minority examples in a two-class imbalanced data task. The proposed method combines cleaning the decision border around minority objects with guided synthetic oversampling. Results of the conducted experimental study indicate that the proposed algorithm usually outperforms the conventional oversampling approaches, especially when the detection of minority examples is considered.

An Imbalanced Data Classification Algorithm of De-noising Auto-Encoder Neural Network Based on SMOTE

Imbalanced data classification problem has always been one of the hot issues in the field of machine learning. Synthetic minority over-sampling technique (SMOTE) is a classical approach to balance datasets, but it may give rise to such problem as noise. Stacked De-noising Auto-Encoder neural network (SDAE), can effectively reduce data redundancy and noise through unsupervised layer-wise greedy learning. Aiming at the shortcomings of SMOTE algorithm when synthesizing new minority class samples, the paper proposed a Stacked De-noising Auto-Encoder neural network algorithm based on SMOTE, SMOTE-SDAE, which is aimed to deal with imbalanced data classification. The proposed algorithm is not only able to synthesize new minority class samples, but it also can de-noise and classify the sampled data. Experimental results show that compared with traditional algorithms, SMOTE-SDAE significantly improves the minority class classification accuracy of the imbalanced datasets.

IFROWANN: Imbalanced Fuzzy-Rough Ordered Weighted Average Nearest Neighbor Classification

Imbalanced classification deals with learning from data with a disproportional number of samples in its classes. Traditional classifiers exhibit poor behavior when facing this kind of data because they do not take into account the imbalanced class distribution. Four main kinds of solutions exist to solve this problem: modifying the data distribution, modifying the learning algorithm for considering the imbalance representation, including the use of costs for data samples, and ensemble methods. In this paper, we adopt the second type of solution and introduce a classification algorithm for imbalanced data that uses fuzzy rough set theory and ordered weighted average aggregation. The proposal considers different strategies to build a weight vector to take into account data imbalance. Our methods are validated by an extensive experimental study, showing statistically better results than 13 other state-of-the-art methods.

Conception of a dominance-based multi-objective local search in the context of classification rule mining in large and imbalanced data sets

Classification on medical data raises several problems such as class imbalance, double meaning of missing data, volumetry or need of highly interpretable results. In this paper a new algorithm is proposed: MOCA-I (Multi-Objective Classification Algorithm for Imbalanced data), a multi-objective local search algorithm that is conceived to deal with these issues all together. It is based on a new modelization as a Pittsburgh multi-objective partial classification rule mining problem, which is described in the first part of this paper. An existing dominance-based multi-objective local search (DMLS) is modified to deal with this modelization. After experimentally tuning the parameters of MOCA-I and determining which version of DMLS algorithm is the most effective, the obtained MOCA-I version is compared to several state-of-the-art classification algorithms. This comparison is realized on 10 small and middle-sized data sets of literature and 2 real data sets; MOCA-I obtains the best results on the 10 data sets and is statistically better than other approaches on the real data sets.

A Novel Selective Ensemble Algorithm for Imbalanced Data Classification Based on Exploratory Undersampling

Learning with imbalanced data is one of the emergent challenging tasks in machine learning. Recently, ensemble learning has arisen as an effective solution to class imbalance problems. The combination of bagging and boosting with data preprocessing resampling, namely, the simplest and accurate exploratory undersampling, has become the most popular method for imbalanced data classification. In this paper, we propose a novel selective ensemble construction method based on exploratory undersampling,RotEasy, with the advantage of improving storage requirement and computational efficiency by ensemble pruning technology. Our methodology aims to enhance the diversity between individual classifiers through feature extraction and diversity regularized ensemble pruning. We made a comprehensive comparison between our method and some state-of-the-art imbalanced learning methods. Experimental results on 20 real-world imbalanced data sets show thatRotEasypossesses a significant increase in performance, contrasted by a nonparametric statistical test and various evaluation criteria.