Imbalanced Data Learning Research Articles

Improving interpolation-based oversampling for imbalanced data learning

The class imbalance problem in machine learning contains a skewed distribution of samples among different classes, resulting in a learning bias toward the majority class. Interpolation-based oversampling techniques solve this problem by yielding the synthetic minority samples to correct the imbalanced class distribution, which has become one of the most popular types of solutions. However, several non-negligible drawbacks exist in them. This study identifies that the flaws of the over constraint, low-efficiency expansion, and over generalization can occur when interpolating the synthetic samples for the inland minority samples (resided in the interior of dense regions of minority class), borderline minority samples (settled in the border of dense regions), and trapped minority samples (located in the sparse regions), respectively. To overcome these flaws, a Position characteristic-Aware Interpolation Oversampling algorithm (PAIO) is proposed. PAIO first leverages a neighborhood-based clustering algorithm to divide the minority samples into three types of samples with different position characteristics (i.e., the inland minority, borderline minority, and trapped minority samples). It then carries out ad hoc interpolation oversampling for these three different types of minority samples, so that the deficiencies can be addressed correspondingly. Moreover, another drawback in interpolation-base oversampling algorithms is the groundless populating for the categorical attributes of synthetic samples. To solve this problem, we develop a Generalized Interpolation Creation way (GIC) to fill the categorical attributes. Extensive experiments on a number of real-world datasets demonstrate that the effectiveness of the proposed PAIO and GIC.

Generative learning for imbalanced data using the Gaussian mixed model

Imbalanced data classification, an important type of classification task, is challenging for standard learning algorithms. There are different strategies to handle the problem, as popular imbalanced learning technologies, data level imbalanced learning methods have elicited ample attention from researchers in recent years. However, most data level approaches linearly generate new instances by using local neighbor information rather than based on overall data distribution. Differing from these algorithms, in this study, we develop a new data level method, namely, generative learning (GL), to deal with imbalanced problems. In GL, we fit the distribution of the original data and generate new data on the basis of the distribution by adopting the Gaussian mixed model. Generated data, including synthetic minority and majority classes, are used to train learning models. The proposed method is validated through experiments performed on real-world data sets. Results show that our approach is competitive and comparable with other methods, such as SMOTE, SMOTE-ENN, SMOTE-TomekLinks, Borderline-SMOTE, and safe-level-SMOTE. Wilcoxon signed rank test is applied, and the testing results show again the significant superiority of our proposal.

A Novel Application of Intelligent Algorithms in Fault Detection of Rudder System

The rudder system is extensively used in aerospace, ships, missiles and other safety demanding areas. Therefore, it is paramount to ensure that the performance of the system is optimal. Rudder system testing equipment is a special tool used to diagnose its failure. Traditional ones can only artificially analyze the massive and complex tested data. Due to the low-test degree of automation, the performance of such testing tools is limited. Aiming to address this shortcoming, we developed a new rudder system testing equipment with four independent loading platforms and intelligent data analysis systems. It sufficiently shortens the installation and commission time of pneumatic actuators and the processing time of the testing data which largely improves its performance and accuracy. Given the imbalanced nature of the data an adaptive sampling algorithm considering informative instances (ASCIN) leveraging the Support Vector Machine (SVM) is proposed to process the originally collected data. The optimal parameters in SVM and ASCIN are searched by Whale Optimization Algorithm (WOA). Experiments are designed to assess the performance of ASCIN in comparison with existing approaches in the area of imbalanced data learning. The results show that the algorithm developed in this study has higher performance relative to traditional approaches. The application of these intelligent algorithms in fault detection and location of rudder system overcomes the limitation of traditional testing equipment and provides a new concept for future research into more intelligent one.

Cost Sensitive Learning and SMOTE Methods for Imbalanced Data

Class imbalance is one of the main problem using different algorithms used in machine learning. In imbalance classification of data the false negative is always high. The researchers have introduced many methods to deal with this problem, but the purpose of this paper is to apply machine learning algorithms under the SMOTE and cost sensitive learning approaches and acquired the results from the different experiments to find out the suitable results for imbalanced data.

Cost Sensitive Learning and SMOTE Methods for Imbalanced Data

Class imbalance is one of the main problem using different algorithms used in machine learning. In imbalance classification of data the false negative is always high. The researchers have introduced many methods to deal with this problem, but the purpose of this paper is to apply machine learning algorithms under the SMOTE and cost sensitive learning approaches and acquired the results from the different experiments to find out the suitable results for imbalanced data.

Ensemble based fuzzy weighted extreme learning machine for gene expression classification

Multi-class imbalance is one of the challenging problems in many real-world applications, from medical diagnosis to intrusion detection, etc. Existing methods for gene expression classification usually assume relatively balanced class distribution. However, the assumption is invalid for imbalanced data learning. This paper presents an effective method named EN-FWELM for class imbalance learning. First, based on a fast classifier extreme learning machine (ELM), fuzzy membership of sample is proposed in order to eliminate classification error coming from noise and outlier samples, and balance factor is introduced in combination with sample distribution and sample number associated with class to alleviate the bias against performance caused by imbalanced data. Furthermore, ensemble of ELMs is used for making classification performance more stable and accurate. A number of base ELMs are removed based on dissimilarity measure, and the remaining base ELMs are integrated by majority voting. Finally, experimental results on various gene expression classification and real-world classification demonstrate that the proposed EN-FWELM remarkably outperforms other approaches in the literature.

Selection-based resampling ensemble algorithm for nonstationary imbalanced stream data learning

Although the issues of concept drift and class imbalance have been studied separately, the joint problem is underexplored even though it has received increasing attention. Concept drift is further complicated when the dataset is class imbalanced. Meanwhile, most of the existing techniques have ignored the influence of complex data distribution on learning imbalanced data streams.To overcome these issues, we propose an ensemble-based model for learning concept drift from imbalanced data streams with complex data distribution, called selection-based resampling ensemble (SRE). SRE combines the operators of resampling and periodical update to handle the joint issue. In the chunk-based framework, a selection-based resampling mechanism, which focuses on drifting and unsafe examples, is first employed to re-balance the class distribution of the latest block. Then, previous ensemble members are periodically updated using the latest examples, where update weights are determined to emphasize costly misclassification examples and minority examples. Meanwhile, SRE can quickly react to new conditions. Empirical studies demonstrate the effectiveness of SRE in learning nonstationary imbalanced data streams.

Fuzzy rule-based oversampling technique for imbalanced and incomplete data learning

Datasets that have skewed class distributions pose a difficulty to learning algorithms in pattern classification. A number of different methods to deal with this problem have been developed in recent years. Specifically, synthetic oversampling techniques focus on balancing the distribution between the training instances of the majority and minority classes by generating extra artificial minority class instances. Unfortunately, few of them can be spread to tackle the problem of imbalanced data with missing values. Moreover, in most cases, existing oversampling methods do not make full use of the correlation between attributes. To this end, in this paper, we propose a fuzzy rule-based oversampling technique (FRO) to handle the class imbalance problem. FRO firstly creates fuzzy rules from the training data and assigns each of them a rule weight, which represents the certainty degree of an instance belonging to the fuzzy subspace. Then it synthesizes new minority instances under the guidance of fuzzy rules. The number of minority instances to be generated under a given fuzzy rule is determined by the rule weight. In a similar way, FRO can also recover the missing values that exist in the imbalanced dataset. Extensive experiments using 55 real-world imbalanced datasets evaluate the performance of the proposed FRO technique. The results show that our method is better than or comparable with a set of alternative state-of-the-art imbalanced classification algorithms in terms of various assessment metrics.

Imbalanced Deep Learning by Minority Class Incremental Rectification.

Model learning from class imbalanced training data is a long-standing and significant challenge for machine learning. In particular, existing deep learning methods consider mostly either class balanced data or moderately imbalanced data in model training, and ignore the challenge of learning from significantly imbalanced training data. To address this problem, we formulate a class imbalanced deep learning model based on batch-wise incremental minority (sparsely sampled) class rectification by hard sample mining in majority (frequently sampled) classes during model training. This model is designed to minimise the dominant effect of majority classes by discovering sparsely sampled boundaries of minority classes in an iterative batch-wise learning process. To that end, we introduce a Class Rectification Loss (CRL) function that can be deployed readily in deep network architectures. Extensive experimental evaluations are conducted on three imbalanced person attribute benchmark datasets (CelebA, X-Domain, DeepFashion) and one balanced object category benchmark dataset (CIFAR-100). These experimental results demonstrate the performance advantages and model scalability of the proposed batch-wise incremental minority class rectification model over the existing state-of-the-art models for addressing the problem of imbalanced data learning.

Imbalanced data fault diagnosis of rotating machinery using synthetic oversampling and feature learning

Imbalanced data problems are prevalent in the real rotating machinery applications. Traditional data-driven diagnosis methods fail to identify the fault condition effectively for lack of enough fault samples. Therefore, this study proposes an effective three-stage fault diagnosis method towards imbalanced data. First, a new synthetic oversampling approach called weighted minority oversampling (WMO) is devised to balance the data distribution. It adopts a new data synthesis strategy to avoid generating incorrect or unnecessary samples. Second, to select useful features automatically, an enhanced deep auto-encoder (DA) approach is adopted. DA is improved in two aspects: 1) a new cost function based on maximum correntropy and sparse penalty is designed to learn sparse robust features; 2) a fine-tuning operation with a self-adaptive learning rate is developed to ensure the good convergence performance. Finally, the C4.5 decision tree identifies the learned features. The proposed method named WMODA is evaluated on 25 benchmark imbalanced datasets. It achieves better results than five well-known imbalanced data learning methods. It is also evaluated on a real engineering dataset. The experimental results show that WMODA can detect more fault samples than the traditional data-driven methods.

Artificial bee colony optimization-based weighted extreme learning machine for imbalanced data learning

The imbalanced datasets are common in real-world application and the problem of imbalanced dataset affect classification performance of many standard learning approaches. To address imbalanced datasets, a weighted extreme learning machine (WELM) solving the L $$_{2}$$ -regularized weighted least squares problem is presented to avoid the generation of an over-fitting model and obtain better generalization ability compared with ELM. However, the weight generated according to class distribution of training data leads to lack of finding optimal weight with good generalization performance and the randomness of input weight and hidden biases of network makes the algorithm produce suboptimal classification model. In this paper, a weighted extreme learning machine based on hybrid artificial bee colony (HABC) is proposed to obtain better performance than WELM, in which input weights and hidden bias of WELM and the weight assigned to training samples are optimized by the hybrid artificial bee colony algorithm. HABC combines the diversities of the perturbed parameter vectors of differential evolution with the best solution information of the artificial bee colony effectively. In the empirical study, different class imbalance data handling methods including four WELM-based methods, weighted support vector machine, four ensemble methods which combine data sampling and the Bagging or Boosting are compared with our method. The experimental results on 15 imbalanced datasets show that the proposed method outperforms most methods, which indicates its superiority.

An Efficient Decision Tree for Imbalance data learning using Confiscate and Substitute Technique

Data mining and knowledge discovery is the process of discovering knowledge from the real world datasets. One of the limitations of the real world datasets is the existence of contamination in the dataset. The existing algorithms performance will degrade due to the contamination in the real world datasets in the form of noisy and missing values. In this paper, we propose a novel algorithm dubbed as Confiscate and Substitute Imbalance Data Learning (CSIDL) for better knowledge discovery from real world datasets. The process of confiscate is implemented in the majority subset for the removal of noisy, border line and missing instances and substitute of missing instances is done in the minority subset for improving the strength of the dataset. Experimental comparisons are done on six real world dataset with bench mark traditional algorithms. The results suggest that the proposed CSIDL algorithm performed better than the compared algorithms in terms of Accuracy, AUC, Precision and F-measure.

Classifying Imbalanced Data Sets by a Novel RE-Sample and Cost-Sensitive Stacked Generalization Method

Learning with imbalanced data sets is considered as one of the key topics in machine learning community. Stacking ensemble is an efficient algorithm for normal balance data sets. However, stacking ensemble was seldom applied in imbalance data. In this paper, we proposed a novel RE-sample and Cost-Sensitive Stacked Generalization (RECSG) method based on 2-layer learning models. The first step is Level 0 model generalization including data preprocessing and base model training. The second step is Level 1 model generalization involving cost-sensitive classifier and logistic regression algorithm. In the learning phase, preprocessing techniques can be embedded in imbalance data learning methods. In the cost-sensitive algorithm, cost matrix is combined with both data characters and algorithms. In the RECSG method, ensemble algorithm is combined with imbalance data techniques. According to the experiment results obtained with 17 public imbalanced data sets, as indicated by various evaluation metrics (AUC, GeoMean, and AGeoMean), the proposed method showed the better classification performances than other ensemble and single algorithms. The proposed method is especially more efficient when the performance of base classifier is low. All these demonstrated that the proposed method could be applied in the class imbalance problem.

A synthetic informative minority over-sampling (SIMO) algorithm leveraging support vector machine to enhance learning from imbalanced datasets

Developing decision support systems (DSS) based on imbalanced datasets is one the critical challenges in data mining and decision-analytics. A dataset is called imbalanced when the number of examples from one class outnumbers the number of the instances from another class. Learning from imbalanced datasets is one of the major challenges in machine learning. While a standard classifier could have a very good performance on a balanced dataset, when applied to an imbalanced dataset, its performance deteriorates dramatically. This poor performance is rather troublesome, especially in detecting the minority class, which usually is the class of interest. Therefore, the poor performance of machine learning techniques, which are used to develop DSS, negatively affect the practicality of DSS in real word problems. Over-sampling the minority class is one of the most promising remedies for imbalanced data learning. In this study, we propose a new synthetic informative minority over-sampling (SIMO) algorithm leveraging support vector machine (SVM). In this algorithm, first SVM is applied to the original imbalanced dataset, then, minority examples close to the SVM decision boundary, as the informative minority examples are over-sampled. We also developed another version of SIMO and call it weighted SIMO (W-SIMO). W-SIMO is different from SIMO in the degree of over-sampling the informative minority examples. In W-SIMO, incorrectly classified informative minority examples are over-sampled with a higher degree compared to the correctly classified informative minority examples. In this way, there is more focus on incorrectly classified minority examples. The over-sampled dataset can be used to train any classifier. We applied these algorithms to the 15 publicly available benchmark imbalanced datasets and assessed their performance in comparison with existing approaches in the area of imbalanced data learning. The results showed that our algorithms had the best performance in all datasets compared to other approaches.

Fuzzy-Based Information Decomposition for Incomplete and Imbalanced Data Learning

Class imbalance and missing values are two critical problems in pattern classification. Researchers have proposed a number of techniques to address each of the problems. However, no single technique can solve the two problems. Moreover, the simple combination approach cannot accurately classify the imbalanced data with missing values. This paper develops a fuzzy-based information decomposition (FID) method to simultaneously address these two problems. In the new FID method, the two different problems are treated as the same missing data estimation problem. In particular, FID rebalances the training data by creating synthetic samples for the minority class. The proposed scheme has two steps: weighting and recovery. In the weighting step, the weights produced by the fuzzy membership functions are used to quantify the contribution of the observed data to the missing estimation. In the recovery step, missing values will be estimated by taking into account different contribution of the observed data. To evaluate the performance of the new FID method, a large number of classification experiments have been carried out on 27 well-known datasets. The results show that the FID method significantly outperforms other ten state-of-the-art individual methods and eight combination methods when missing values and imbalanced data present at the same time.

Synthetic minority oversampling technique for multiclass imbalance problems

Multiclass imbalance data learning has attracted increasing interests from the research community. Unfortunately, existing oversampling solutions, when facing this more challenging problem as compared to two-class imbalance case, have shown their respective deficiencies such as causing serious over generalization or not actively improving the class imbalance in data space. We propose a k-nearest neighbors (k-NN)-based synthetic minority oversampling algorithm, termed SMOM, to handle multiclass imbalance problems. Different from previous k-NN-based oversampling algorithms, where for any original minority instance the synthetic instances are randomly generated in the directions of its k-nearest neighbors, SMOM assigns a selection weight to each neighbor direction. The neighbor directions that can produce serious over generalization will be given small selection weights. This way, SMOM forms a mechanism of avoiding over generalization as the safer neighbor directions are more likely to be selected to yield the synthetic instances. Owing to this, SMOM can aggressively explore the regions of minority classes by configuring a high value for parameter k, but do not result in severe over generalization. Extensive experiments using 27 real-world data sets demonstrate the effectiveness of our algorithm.

Distributed and weighted extreme learning machine for imbalanced big data learning

The Extreme Learning Machine (ELM) and its variants are effective in many machine learning applications such as Imbalanced Learning (IL) or Big Data (BD) learning. However, they are unable to solve both imbalanced and large-volume data learning problems. This study addresses the IL problem in BD applications. The Distributed and Weighted ELM (DW-ELM) algorithm is proposed, which is based on the MapReduce framework. To confirm the feasibility of parallel computation, first, the fact that matrix multiplication operators are decomposable is illustrated. Then, to further improve the computational efficiency, an Improved DW-ELM algorithm (IDW-ELM) is developed using only one MapReduce job. The successful operations of the proposed DW-ELM and IDW-ELM algorithms are finally validated through experiments.

A Novel Ensemble Method for Imbalanced Data Learning: Bagging of Extrapolation-SMOTE SVM.

Class imbalance ubiquitously exists in real life, which has attracted much interest from various domains. Direct learning from imbalanced dataset may pose unsatisfying results overfocusing on the accuracy of identification and deriving a suboptimal model. Various methodologies have been developed in tackling this problem including sampling, cost-sensitive, and other hybrid ones. However, the samples near the decision boundary which contain more discriminative information should be valued and the skew of the boundary would be corrected by constructing synthetic samples. Inspired by the truth and sense of geometry, we designed a new synthetic minority oversampling technique to incorporate the borderline information. What is more, ensemble model always tends to capture more complicated and robust decision boundary in practice. Taking these factors into considerations, a novel ensemble method, called Bagging of Extrapolation Borderline-SMOTE SVM (BEBS), has been proposed in dealing with imbalanced data learning (IDL) problems. Experiments on open access datasets showed significant superior performance using our model and a persuasive and intuitive explanation behind the method was illustrated. As far as we know, this is the first model combining ensemble of SVMs with borderline information for solving such condition.

A Critique of Imbalanced Data Learning Approaches for Big Data Analytics

Biomedical research becomes reliant on multi-disciplinary, multi-institutional collaboration, and data sharing is becoming increasingly important for researchers to reuse experiments, pool expertise and validate approaches. However, there are many hurdles for data sharing, including the unwillingness to share, lack of flexible data model for providing context information for shared data, difficulty to share syntactically and semantically consistent data across distributed institutions, and expensive cost to provide tools to share the data. In our work, we develop a web-based collaborative biomedical data sharing platform SciPort to support biomedical data sharing across distributed organisations. SciPort provides a generic metadata model for researchers to flexibly customise and organise the data. To enable convenient data sharing, SciPort provides a central server-based data sharing architecture, where data can be shared by one click through publishing metadata to the central server. To enable consistent data sharing, SciPort provides collaborative distributed schema management across distributed sites. To enable semantic consistency for data sharing, SciPort provides semantic tagging through controlled vocabularies. SciPort is lightweight and can be easily deployed for building data sharing communities for biomedical research.

Evaluation of Cost Sensitive Learning for Imbalanced Bank Direct Marketing Data

Objectives: The imbalanced bank direct marketing data set utilized in this study is a two-class data mining problem, where a customer may or may not subscribe a product from a bank. Methods/Statistical Analysis: The data set inherited the rare class problem where the classification rate attained for the rare class is low. In this study, we attempted cost sensitive learning to mitigate the problem, and to address that there are various costs involved when misclassification occurs. Three learning algorithms, namely, Naive Bayes (NB), C4.5 and Naive Bayes Tree (NBT) were involved in the cost sensitive learning and their results were empirically evaluated. Findings: The results were also compared with two previous studies that utilized the cost insensitive SVM and over-sampling, respectively. Although cost sensitive learning is claimed able to handle imbalanced data sets, but we noticed that the learning is less effective for the bank direct marketing data set in overall. Cost sensitive learning provides a way of “wrapping” learning algorithms that are not designed to handle imbalanced class distributions. Therefore, it may not work well for certain imbalanced data sets. Over-sampling, on the other hand, worked well for the data set. Improvements/Applications: Over-sampling helped to generalize the decision region of the rare class clearly and subsequently improved the classification result. Keywords: Bank Direct Marketing, Cost Sensitive Learning, Imbalanced Data Set, Rare Class Problem, Over-Sampling