Multi-class Imbalanced Learning Research Articles

AMDO: An Over-Sampling Technique for Multi-Class Imbalanced Problems

Multi-class imbalanced problems have attracted growing attention from the real-world classification tasks in engineering. The underlying skewed distribution of multiple classes poses difficulties for learning algorithms, which becomes more challenging when considering overlapping between classes, lack of representative data, and mixed-type data. In this work, we address this problem in a data-oriented way. Motivated by a recently proposed over-sampling technique designed for numeric data sets, Mahalanobis Distance-based Over-sampling (MDO), we use this technique to capture the covariance structure of the minority class and to generate synthetic samples along the probability contours for learning algorithms. Based on MDO, we further improve the over-sampling strategy and generalize it for mixed-type data sets. The established technique, Adaptive Mahalanobis Distance-based Over-sampling (AMDO), introduces GSVD (Generalized Singular Value Decomposition) for mixed-type data, develops a partially balanced resampling scheme and optimizes the sample synthesis. Theoretical analysis is conducted to demonstrate the reasonability of AMDO. Extensive experimental testing is performed on 15 multi-class imbalanced benchmarks and two data sets for precipitation phase recognition in comparison with several state-of-the-art multi-class imbalanced learning methods. The results validate the effectiveness and robustness of our proposal.

An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme

Class-imbalance learning is one of the most challenging problems in machine learning. As a new and important direction in this field, multi-class imbalanced data classification has attracted a great many research focus in recent years. In this paper, we first make a very comprehensive review on state-of-the-art classification algorithms for multi-class imbalanced data. Moreover, we propose a new multi-class imbalance classification algorithm, which is hereafter referred to as the Diversified Error Correcting Output Codes (DECOC) method. The main idea of DECOC is to combine the improved ECOC (Error Correcting Output Codes) method for tackling class imbalance, and the diversified ensemble learning framework, which finds the best classification algorithm (out of many heterogeneous classification algorithms) for each individual sub-dataset resampled from the original data. We conduct experiments on 19 public datasets to empirically compare the performance of DECOC with 17 state-of-the-art multi-class imbalance learning algorithms, using 4 different accuracy measures: overall accuracy, Geometric mean, F-measure, and Area Under Curve. Experimental results demonstrate that DECOC achieves significantly better accuracy performance than the other 17 algorithms on these accuracy metrics. To advance research in this field, we make all the source codes of DECOC and the above-mentioned 17 state-of-the-art algorithms for imbalanced data classification be available at GitHub: https://github.com/chongshengzhang/Multi_Imbalance.

Postboosting Using Extended G-Mean for Online Sequential Multiclass Imbalance Learning.

In this paper, a novel learning method called postboosting using extended G-mean (PBG) is proposed for online sequential multiclass imbalance learning (OS-MIL) in neural networks. PBG is effective due to three reasons. 1) Through postadjusting a classification boundary under extended G-mean, the challenging issue of imbalanced class distribution for sequentially arriving multiclass data can be effectively resolved. 2) A newly derived update rule for online sequential learning is proposed, which produces a high G-mean for current model and simultaneously possesses almost the same information of its previous models. 3) A dynamic adjustment mechanism provided by extended G-mean is valid to deal with the unresolved challenging dense-majority problem and two dynamic changing issues, namely, dynamic changing data scarcity (DCDS) and dynamic changing data diversity (DCDD). Compared to other OS-MIL methods, PBG is highly effective on resolving DCDS, while PBG is the only method to resolve dense-majority and DCDD. Furthermore, PBG can directly and effectively handle unscaled data stream. Experiments have been conducted for PBG and two popular OS-MIL methods for neural networks under massive binary and multiclass data sets. Through the analyses of experimental results, PBG is shown to outperform the other compared methods on all data sets in various aspects including the issues of data scarcity, dense-majority, DCDS, DCDD, and unscaled data.

DRCW-ASEG: One-versus-One distance-based relative competence weighting with adaptive synthetic example generation for multi-class imbalanced datasets

Abstract Multi-class imbalance learning problems suffering from the different distribution of classes occur in many real-world applications. One-versus-One (OVO) decomposition strategy is a common and useful technique used to address multi-class classification problems, which consists in dividing the original multi-class problem into all binary class sub-problems. The effort to reduce the effect of non-competent classifiers has proven to be a useful way of improving the performance in the OVO scheme. However, these approaches might not be effective for imbalance scenarios, since they are based on standard biased learning procedures. On this account, we propose a novel approach named Distance-based Relative Competence Weighting with Adaptive Synthetic Example Generation (DRCW-ASEG), which properly addresses the synergy between imbalance learning and dynamic classifier weighting in OVO scheme. This new proposed algorithm aims to dynamically produce synthetic examples of minority classes in the stage of dynamic weighting process. We develop a thorough experimental study in order to verify the benefits of the proposed algorithm considering different base binary classifiers.

An Empirical Comparison of Classifiers for Multi-Class Imbalance Learning

This paper presents empirical analysis of binarisation solutions to multi-class problems such as one-vs-one (OVO), one-vs-rest (OVR), error correcting output code (ECOC) that make use of Linear SVM as a base learner and other ensemble-based solutions like Bagging, Random Forest with decision tree as the base learner. The data sets used for experiments are multi-class and imbalanced in nature and have been imported from UCI machine learning repository. Due to the imbalanced nature of datasets, we have evaluated all the multi-class algorithms using the performance metrics such as F1-Score, Precision, Recall and Gmean. And then we compared all the algorithms over all the benchmark datasets using Friedman's statistical test. We have also analysed the impact of non-intelligent sampling techniques on these algorithms. Sci-Kit, imbalanced-learn machine learning tool kits were used to carry out the experiments in this paper.

An Integrated Ensemble Learning Model for Imbalanced Fault Diagnostics and Prognostics

With the development of artificial intelligence technology, data-driven fault diagnostics and prognostics in industrial systems have been a hot research area since the large volume of industrial data is being collected from the industrial process. However, imbalanced distributions exist pervasively between faulty and normal samples, which leads to imprecise fault diagnostics and prognostics. In this paper, an effective imbalance learning algorithm Easy-SMT is proposed. Easy-SMT is an integrated ensemble-based method, which comprises synthetic minority oversampling technique (SMOTE)-based oversampling policy to augment minority faulty classes and EasyEnsemble to transfer an imbalanced class learning problem into an ensemble-based balanced learning subproblem. We validate the feasibility and effectiveness of the proposed method in a real wind turbine failure forecast challenge, and our solution has won the third place among hundreds of teams. Moreover, we also evaluate the method on prognostics and health management 2015 challenge datasets, and the results show that the model could also achieve good performance on multiclass imbalance learning task compared with baseline classifiers.

Multiclass imbalance learning: Improving classification of pediatric brain tumors from magnetic resonance spectroscopy.

PurposeClassification of pediatric brain tumors from 1H‐magnetic resonance spectroscopy (MRS) can aid diagnosis and management of brain tumors. However, varied incidence of the different tumor types leads to imbalanced class sizes and introduces difficulties in classifying rare tumor groups. This study assessed different imbalanced multiclass learning techniques and compared the use of complete spectra and quantified metabolite profiles for classification of three main childhood brain tumor types.MethodsSingle‐voxel, Short echo time MRS data were collected from 90 patients with pilocytic astrocytoma (n = 42), medulloblastoma (n = 38), or ependymoma (n = 10). Both spectra and metabolite profiles were used to develop the learning algorithms. The borderline synthetic minority oversampling technique and AdaboostM1 were used to correct for the skewed distribution. Classifiers were trained using five different pattern recognition algorithms.ResultsUse of imbalanced learning techniques improved the balanced accuracy rate (BAR) of all classification methods (average BAR over all classification methods for spectra: oversampled data = 0.81, original = 0.63, P < 0.001; metabolite concentration: oversampled‐data = 0.91, original = 0.75, P < 0.0001). Performance of all classifiers in discriminating ependymomas increased when oversampled data were used compared with original data for both complete spectra (F‐measure P < 0.01) and metabolite profile (F‐measure P < 0.001).ConclusionImbalanced learning techniques improve the classification accuracy of childhood brain tumors from MRS where group sizes differ and facilitate the inclusion of rarer tumor types into clinical decision support systems. Magn Reson Med 77:2114–2124, 2017. © 2016 The Authors Magnetic Resonance in Medicine published by Wiley Periodicals, Inc. on behalf of International Society for Magnetic Resonance in Medicine. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Various Approaches for Multiclass Imbalance Learning Issues with MLP

Imbalance data is a major issue, which can be either binary or multiclass. Oversampling, Undersampling, SMOTE, SMOTEboost, Adaboost, OSS (One Sided Selection) and many other algorithms are there to deal with binary or multiclass imbalance issues. Software Defect Predictors (SDPs) and Software Cost Estimations (SCEs) are tools that used to classify the software elements into certain factors which helps in studying the imbalance problem. First take into consideration the SDPs to predict the defect prone part of software so that project can be completed with expected quality. In the same way for SCEs, certain factors will need to be considered for overall cost estimation in a way that financials can be managed and software elements can be done neatly. Class imbalance learning challenges, supervised learning difficulties where some classes have significantly more samples than others, i.e. dataset having a set of majority and minority samples. To make imbalance data balanced, most of the present study focused only on binary-class cases. In this paper, Adaboost.NC method is introduced and its result will be analyzed with proposed Dynamic Sampling methodMultilayer Perceptrons ((DyS)-MLP) for multiclass imbalance problem. General Terms Information retrieval through Data Mining.

Adapted ensemble classification algorithm based on multiple classifier system and feature selection for classifying multi-class imbalanced data

Learning from imbalanced data, where the number of observations in one class is significantly rarer than in other classes, has gained considerable attention in the data mining community. Most existing literature focuses on binary imbalanced case while multi-class imbalanced learning is barely mentioned. What's more, most proposed algorithms treated all imbalanced data consistently and aimed to handle all imbalanced data with a versatile algorithm. In fact, the imbalanced data varies in their imbalanced ratio, dimension and the number of classes, the performances of classifiers for learning from different types of datasets are different. In this paper we propose an adaptive multiple classifier system named of AMCS to cope with multi-class imbalanced learning, which makes a distinction among different kinds of imbalanced data. The AMCS includes three components, which are, feature selection, resampling and ensemble learning. Each component of AMCS is selected discriminatively for different types of imbalanced data. We consider two feature selection methods, three resampling mechanisms, five base classifiers and five ensemble rules to construct a selection pool, the adapting criterion of choosing each component from the selection pool to frame AMCS is analyzed through empirical study. In order to verify the effectiveness of AMCS, we compare AMCS with several state-of-the-art algorithms, the results show that AMCS can outperform or be comparable with the others. At last, AMCS is applied in oil-bearing reservoir recognition. The results indicate that AMCS makes no mistake in recognizing characters of layers for oilsk81-oilsk85 well logging data which is collected in Jianghan oilfield of China.