Multi-class Imbalanced Datasets Research Articles

Feature selection and its combination with data over-sampling for multi-class imbalanced datasets

Feature selection aims at filtering out some unrepresentative features from a given dataset in order to construct more effective learning models. Furthermore, ensemble feature selection by combining multiple feature selection methods has shown its outperformance over single feature selection. However, the performances of different (ensemble) feature selection methods have not been fully examined over multi-class imbalanced datasets. On the other hand, for class imbalanced datasets, one widely considered solution is to re-balance the datasets by data over-sampling, which generates some synthetic examples for the minority classes. However, the effect of performing (ensemble) feature selection on over-sampling multi-class imbalanced datasets has not been investigated. Therefore, the first research objective is to examine the performances of single and ensemble feature selection methods by fifteen well-known filter, wrapper, and embedded algorithms in terms of classification accuracy. For the second research objective, two orders of combining the feature selection and over-sampling steps are compared in order to find out the best combination procedure as well as the best combined algorithms. The experimental results based on ten different domain datasets containing low to very high feature dimensions show that ensemble feature selection methods slightly perform better than single ones. However, their performance differences are not big. To combine with the Synthetic Minority Oversampling Technique (SMOTE) over-sampling algorithm, performing feature selection first and over-sampling second outperforms the other procedure. Although the best combined algorithms are based on ensemble feature selection, eXtreme Gradient Boosting (XGBoost), as the single best feature selection algorithm, combined with SMOTE provides very similar classification performance to the best combined algorithms. To consider the issues of classification performance and compactional cost, the optimal solution is based on the combined XGBoost and SMOTE.

To Combat Multiclass Imbalanced Problems by Aggregating Evolutionary Hierarchical Classifiers.

Real-world datasets are often imbalanced, posing frequent challenges to canonical machine learning algorithms that assume a balanced class distribution. Moreover, the imbalance problem becomes more complicated when the dataset is multiclass. Although many approaches have been presented for imbalanced learning (IL), research on the multiclass imbalanced problem is relatively limited and deficient. To alleviate these issues, we propose a forest of evolutionary hierarchical classifiers (FEHC) method for multiclass IL (MCIL). FEHC can be seen as a classifier fusion framework with a forest structure, and it aggregates several evolutionary hierarchical multiclassifiers (EHMCs) to reduce generalization error. Specifically, a multichromosome genetic algorithm (MCGA) is designed to simultaneously select (sub)optimal features, classifiers, and hierarchical structures when generating these EHMCs. The MCGA adopts a dynamic weighting module to learn difficult classes and promote the diversity of FEHC. We also present the "stratified underbagging" (SUB) strategy to address class imbalance and the "soft tree traversal" (STT) strategy to make FEHC converge faster and better. We thoroughly evaluate the proposed algorithm using 14 multiclass imbalanced datasets with various properties. Compared with popular and state-of-the-art approaches, FEHC obtains better performance under different evaluation metrics. Codes have been made publicly available on GitHub.https://github.com/CUHKSZ-NING/FEHCClassifier.

Imbalanced least squares regression with adaptive weight learning

Least squares regression (LSR) has demonstrated promising performance in various classification tasks owing to its effectiveness and efficiency. However, there are some deficiencies that seriously hinder its application in imbalanced data scenarios. The first is that LSR strongly relies on a balanced class distribution. A severely imbalanced class distribution may seriously damage the effectiveness of the algorithm. Second, the utilized binary label matrix in the conventional LSR model may be too strict to learn a discriminative transformation matrix for imbalanced learning. To address the above issues, in this paper, an adaptive weight learning mechanism and label relaxation constraint are proposed and incorporated into the framework of LSR to tackle the imbalanced classification problem. The weight of each sample can be adaptively obtained according to the original distribution information of the imbalanced data, in which the importance of minority class samples can be better reflected with larger weights. A new label relaxation matrix consisting of the original label matrix and auxiliary matrix is constructed to widen the margins between different classes. Further, we provide an iterative algorithm with fast convergence to solve the resulting optimization problem. Extensive experimental results on diverse binary-class and multi-class imbalanced datasets show that the proposed method outperforms many other state-of-the-art imbalanced learning approaches.

An Oversampling Mechanism for Multimajority Datasets using SMOTE and Darwinian Particle Swarm Optimisation

Data skewness continues to be one of the leading factors which adversely impacts the machine learning algorithms performance. An approach to reduce this negative effect of the data variance is to pre-process the former dataset with data level resampling strategies. Resampling strategies have been seen in two forms, oversampling and undersampling. An oversampling strategy is proposed in this article for tackling multiclass imbalanced datasets. This proposed approach optimises the state-of-the-art oversampling technique SMOTE with the Darwinian Particle Swarm Optimization technique. This proposed method DOSMOTE generates synthetic optimised samples for balancing the datasets. This strategy will be more effective on multimajority datasets. An experimental study is performed on peculiar multimajority datasets to measure the effectiveness of the proposed approach. As a result, the proposed method produces promising results when compared to the conventional oversampling strategies.

Hybrid Approach with Distance Feature for Multi-Class Imbalanced Datasets

The multi-class imbalance problem has a higher level of complexity when compared to the binary class problem. The difficulty is due to the large number of classes that will present challenges related to overlapping between classes. Many approaches have been proposed to deal with these multi-class problems. One is a hybrid approach combining a data-level approach and an algorithm-level approach. This approach is done by the ensemble on the classifier and also oversampling on the minority class. SMOTE is an oversampling method that provides good performance, but this method is necessary to determine the best sample used in the interpolation process to generate new samples. The need for determining the best sample is related to the overlap between classes that always accompanies the multi-class imbalance problem. The existence of overlap requires efforts to determine the safe region to synthesize the sample in the oversampling process in SMOTE. The safe region is considered the best for synthesizing samples due to the lower tendency of overlapping. It can be done by constructing distance features to determine the safe region. The sample with the best distance and the lowest imbalance ratio will be selected as a sample in the over-sampling process with SMOTE. The main contribution of this research is the proposed method of Hybrid Approach with Distance Feature so that it can determine safe samples, with the main advantage being in addition to handling multi-class imbalances, it is also better for handling overlapping. The results of this study will be compared with Multiple Random Balance (MultiRandBal) which performs a random oversampling process. The results showed that the Augmented R-Value, Class Average Accuracy, Class Balance Accuracy, and Hamming Loss obtained in this method was better than the random oversampling process. These results also show that the Hybrid Approach with Distance Feature provides better results in handling multi-class imbalances when compared to MultiRandBal.

Dimensionality reduction techniques in structural and earthquake engineering

Dimensionality reduction (DR) techniques are becoming increasingly important in engineering and scientific applications for feature extraction and data visualization purposes. This problem gets significantly more demanding when encountering class-imbalanced data sets with a disproportionate ratio of observations in each class. The main objective of this paper is to thoroughly evaluate the performance of various dimensionality reduction techniques for analyzing a cognate set of computational and experimental data in the field of structural and earthquake engineering with varying numbers of classes. Notably, a data set is generated using a computer simulation that characterizes risks posed by earthquakes to structures. Depending on the severity of damage in the simulations, data is classified into three, five, and eight classes. This research study also uses three existing multi-class imbalanced data sets that have been developed independently. For each problem, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and t-Distributed Stochastic Neighbor Embedding (t-SNE) are used, to cover a wide range of linear/non-linear and supervised/unsupervised DR techniques. To overcome the class-imbalance problem, the efficacy of the Synthetic Minority Oversampling (SMOTE) technique considering various classifiers, including K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Trees (DT), Random Forest (RF), MLPClassifier (MLP), and AdaBoost are investigated. It is observed that combining SMOTE and LDA shows promising results for training classifiers on the reduced data. In light of the findings of this study, it is recommended to train DT classifiers after reducing the dimensionality of the input data. It is through this process that the scientific data is transformed into a two-dimensional space for visualizing decision surfaces, which is helpful for practitioners to understand the performance of classifiers.

Novel hybrid classification model for multi-class imbalanced lithology dataset

Deep learning methods and applications assist geologists in predicting and identifying lithologies in different surveys, hence lowering operational costs and uptime. This allows accurate data analysis and completion of scientific research on data obtained at geologically different places. This study used 4 lithologies datasets with high dimensionality and multiclass imbalance problems for analysis and classification. The imbalance of data classification is one of the most important problems facing current data analysis. Data imbalance can considerably influence classification performance, especially when dealing with other difficulty variables such as the presence of overlapping class distributions. This impact is especially obvious in multi-class conditions when mutual imbalance relations across classes complicate matters even further. Moreover, the problem of high dimensionality can lead to increased computing complexity and overfitting, and thus these issues can affect classification performance. To overcome these problems, we developed a new hybrid deep learning multi-class imbalanced learning method that combines Synthetic Minority Oversampling (SMOTE) to resample the data, and Recursive Feature Elimination (RFE) to identify the most useful predictive features. Finally, we believe that our developments can help improve geology research by providing accurate classification and rapid answers about interpreting data obtained in various study areas.

Multi-class imbalanced enterprise credit evaluation based on asymmetric bagging combined with light gradient boosting machine

Most existing research on multi-class imbalanced enterprise credit evaluation modeling has been built on data-level imbalance processing methods and single classifier approaches. Using the one-versus-one (OVO) decomposition and fusion method to dispose multi-class classification, this paper proposes two new credit evaluation ensemble models by combining the asymmetric bagging (AB) and the light gradient boosting machine (LightGBM). Based on a multi-class imbalanced dataset of Chinese enterprises that issue corporate bonds from 2014 through 2020, this study conducts a series of empirical experiments for multi-class imbalanced enterprise credit evaluation. The experimental results demonstrate that our proposed models can significantly outperform the benchmark models, which integrate the OVO and one-versus-all decomposition and fusion method respectively with the random under sampling LightGBM, random over sampling LightGBM, synthetic minority over sampling technique LightGBM and the AB decision tree. In addition, the proposed models can carry out analysis on feature importance, which provides decision-making basis for enterprise stakeholders.

Diabetic retinopathy screening using deep learning for multi-class imbalanced datasets

Screening and diagnosis of diabetic retinopathy disease is a well known problem in the biomedical domain. The use of medical imagery from a patient’s eye for detecting the damage caused to blood vessels is a part of the computer-aided diagnosis that has immensely progressed over the past few years due to the advent and success of deep learning. The challenges related to imbalanced datasets, inconsistent annotations, less number of sample images and inappropriate performance evaluation metrics has caused an adverse impact on the performance of the deep learning models. In order to tackle the effect caused by class imbalance, we have done extensive comparative analysis between various state-of-the-art methods on three benchmark datasets of diabetic retinopathy: - Kaggle DR detection, IDRiD and DDR, for classification, object detection and segmentation tasks. This research could serve as a concrete baseline for future research in this field to find appropriate approaches and deep learning architectures for imbalanced datasets.

Unsupervised novelty pattern classification of shmoo plots for visualizing the test results of integrated circuits

Shmoo plots are visual tools for verifying the performance of integrated circuit devices, where each cell in a plot records whether the examined device operates normally under the test condition. When identifying the device state, the overall pass/fail pattern appearing in the shmoo plot is more important than the test results for individual conditions. Because similar shmoo plots indicate similar device characteristics, defect causes, and process peculiarities, engineers can analyze device quality and defect causes by classifying shmoo plot patterns. Most mass-produced devices have high and stable yields, whereas defect devices are incredibly scarce. If engineers classify numerous device plots manually at a semiconductor test site, significant time and resources will be required, and the result will likely vary based on the engineers’ experience. Therefore, shmoo plot usage is limited unless an automatic classification model is adopted. Moreover, training high-performance pattern classifiers that do not overfit the models is difficult because shmoo plots contain high-dimensional data and unlabeled, multiclass imbalanced datasets, where the number of defects is smaller than that of normal plots and pattern labels are seldom assigned. In this study, we propose a novel feature extraction process and a two-stage clustering process to distinguish novel shmoo plot patterns. Actual shmoo plots obtained from a wafer test stage are used to compare the experimental results obtained via the proposed method and conventional methods, and they verify the superiority of the proposed method.

Multi-class WHMBoost: An ensemble algorithm for multi-class imbalanced data

The imbalanced data problem is widespread in the real world. In the process of training machine learning models, ignoring imbalanced data problems will cause the performance of the model to deteriorate. At present, researchers have proposed many methods to deal with the imbalanced data problems, but these methods mainly focus on the imbalanced data problems in two-class classification tasks. Learning from multi-class imbalanced data sets is still an open problem. In this paper, an ensemble method for classifying multi-class imbalanced data sets is put forward, called multi-class WHMBoost. It is an extension of WHMBoost that we proposed earlier. We do not use the algorithm used in WHMBoost to process the data, but use random balance based on average size so as to balance the data distribution. The weak classifiers we use in the boosting algorithm are support vector machine and decision tree classifier. In the process of training the model, they participate in training with given weights in order to complement each other’s advantages. On 18 multi-class imbalanced data sets, we compared the performance of multi-class WHMBoost with state of the art ensemble algorithms using MAUC, MG-mean and MMCC as evaluation criteria. The results demonstrate that it has obvious advantages compared with state of the art ensemble algorithms and can effectively deal with multi-class imbalanced data sets.

Health evaluation methodology of remote maintenance control system of natural gas pipeline based on ACWGAN-GP algorithm

The remote maintenance system of natural gas pipeline is of great significance to ensure the safe and stable operation of the pipeline network. The multi-classification method based on machine learning is more effective for the health evaluation of remote maintenance control system than the traditional evaluation method based on expert experience. In view of the sev ere imb alance i n the nu mber of s amples of f ive health levels, a health evaluation methodology of remote maintenance control system based on Wasserstein distance sand auxiliary classification generative adversarial network (ACWGAN-GP) is proposed. Firstly, the model stability is improved by introducing Wasserstein distance and gradient penalty. The generator generates balanced data, while the discriminator trains with generated and actual data. In this way, several ACWGAN-GP sub-models are trained. Then, the health levels of the sub-model are directly obtained by using the discriminator to classify the samples. Finally, according to the hierarchical relationship of the system, a parallel-serial combined evaluation method is adopted. By this means, the health evaluation model of remote maintenance control system including ACWGAN-GP sub-models is constructed. The experimental results based on 13 sets of KEEL and UCI multi-class imbalanced datasets and actual sampling data show that the effectiveness and advancement of the proposed method improved significantly compared with the existing similar typical algorithms.

The Effect of Adaptive Synthetic and Information Gain on C4.5 and Naive Bayes in Imbalance Class Dataset

Class imbalance is a severe problem in classification due to the deep slope on the class axis. The dataset is dominated by the majority class, which has the potential for misclassification. Another problem in classification and clustering is that high-dimensional datasets are found that have the potential to affect the performance of classification algorithms in terms of computation and accuracy. In this study, the class imbalance was handled using the ADASYN k - NN resampling technique and the selection feature using Information Gain. Based on the evaluation results, the sampling contribution matrix can improve the classification model by improving the geometric mean value. The selection feature helps interpret data with more simple features but can reduce the accuracy of the results. The results showed that the implementation of ADASYN k-NN and Information Gain could increase the accuracy score and geometric mean score of Decision Tree C4.5 and Naive Bayes. For further work, this proposed method will be tested on multiclass imbalanced datasets.

Performance enrichment through parameter tuning of random forest classification for imbalanced data applications

One of the foremost application domains in today’s real-life scenario is unequal data distribution within datasets, related classifiers, and its Performance Enrichment Techniques (PET). Random Forest Classification (RFC) is one of the most efficient techniques that can function speedily over binary or multiclass imbalanced characteristics datasets. With its built-in ensemble capacity, building a generalized model on any Binary Imbalanced Datasets (BID) and Multiclass Imbalanced Datasets (MID) gets much easier. Related work carried out here implies that the attention of researchers is inclined towards ID applications and related machine learning techniques. RFC gives improvised performance due to Ensemble Approach (EA). EA generates several classifiers and segregates the results as PET. The performance of the single classifier is lower than the performance of the set of multiple classifiers. The associated study in this research paper provides insight into three PET, which are the count of folds in cross-validation, variable count of trees and the feature selection method through Regression Analysis (RA) and it’s tuning with RFC Classifier. Time and Cost required for classification is saved with the help of RA. Extensive experimental analysis carried out with a predictive modeler Salford System with proposed PET and tested on 32 benchmark datasets, which justifies the enhanced performance of RFC.

Performance Analysis of Two-Stage Iterative Ensemble Method over Random Oversampling Methods on Multiclass Imbalanced Datasets

Data imbalance occurring among multiclass datasets is very common in real-world applications. Existing studies reveal that various attempts were made in the past to overcome this multiclass imbalance problem, which is a severe issue related to the typical supervised machine learning methods such as classification and regression. But, still there exists a need to handle the imbalance problem efficiently as the datasets include both safe and unsafe minority samples. Most of the widely used oversampling techniques like SMOTE and its variants face challenges in replicating or generating the new data instances for balancing them across multiple classes, particularly when the imbalance is high and the number of rare samples count is too minimal thus leading the classifier to misclassify the data instances. To lessen this problem, we proposed a new data balancing method namely a two-stage iterative ensemble method to tackle the imbalance in multiclass environment. The proposed approach focuses on the rare minority sample’s influence on learning from imbalanced datasets and the main idea of the proposed approach is to balance the data without any change in class distribution before it gets trained by the learner such that it improves the learner’s learning process. Also, the proposed approach is compared against two widely used oversampling techniques and the results reveals that the proposed approach shows a much significant improvement in the learning process among the multiclass imbalanced data.

Gene Selection in Multi-class Imbalanced Microarray Datasets Using Dynamic Length Particle Swarm Optimization

Background: Microarray gene expression datasets usually contain a large number of genes that complicate further operations like classification, clustering and other kinds of analysis. During the classification process, the identification of salient genes is a brainstorming task and needs a careful selection. Methods: The classification of multi-class datasets is more critical when compared with binary classification. When there are multiple class labels, chances are more likely that the datasets are imbalanced. Large variations can be seen in the number of samples belonging to each class, and hence the classification process may go biased with incorrect samples chosen for training. There is no sufficient research work available to address all these three scenarios together in microarray datasets. Results and Discussion: The paper fills this gap with the following contributions: i) Selects salient genes for classification using multiSURF algorithm ii) Identifies right instances from imbalanced datasets using Retained Tomek Link algorithm and iii) Performs gene selection for multi-class classification using Dynamic Length Particle Swarm Optimization (DPSO). Conclusion: The proposed method is implemented on multi-class imbalanced microarray datasets, and the final classification performance is seen to be encouraging and better than other compared methods.

Scalable kernel-based SVM classification algorithm on imbalance air quality data for proficient healthcare

In the last decade, we have seen drastic changes in the air pollution level, which has become a critical environmental issue. It should be handled carefully towards making the solutions for proficient healthcare. Reducing the impact of air pollution on human health is possible only if the data is correctly classified. In numerous classification problems, we are facing the class imbalance issue. Learning from imbalanced data is always a challenging task for researchers, and from time to time, possible solutions have been developed by researchers. In this paper, we are focused on dealing with the imbalanced class distribution in a way that the classification algorithm will not compromise its performance. The proposed algorithm is based on the concept of the adjusting kernel scaling (AKS) method to deal with the multi-class imbalanced dataset. The kernel function's selection has been evaluated with the help of weighting criteria and the chi-square test. All the experimental evaluation has been performed on sensor-based Indian Central Pollution Control Board (CPCB) dataset. The proposed algorithm with the highest accuracy of 99.66% wins the race among all the classification algorithms i.e. Adaboost (59.72%), Multi-Layer Perceptron (95.71%), GaussianNB (80.87%), and SVM (96.92). The results of the proposed algorithm are also better than the existing literature methods. It is also clear from these results that our proposed algorithm is efficient for dealing with class imbalance problems along with enhanced performance. Thus, accurate classification of air quality through our proposed algorithm will be useful for improving the existing preventive policies and will also help in enhancing the capabilities of effective emergency response in the worst pollution situation.

Experimental Study and Comparison of Imbalance Ensemble Classifiers with Dynamic Selection Strategy.

Imbalance ensemble classification is one of the most essential and practical strategies for improving decision performance in data analysis. There is a growing body of literature about ensemble techniques for imbalance learning in recent years, the various extensions of imbalanced classification methods were established from different points of view. The present study is initiated in an attempt to review the state-of-the-art ensemble classification algorithms for dealing with imbalanced datasets, offering a comprehensive analysis for incorporating the dynamic selection of base classifiers in classification. By conducting 14 existing ensemble algorithms incorporating a dynamic selection on 56 datasets, the experimental results reveal that the classical algorithm with a dynamic selection strategy deliver a practical way to improve the classification performance for both a binary class and multi-class imbalanced datasets. In addition, by combining patch learning with a dynamic selection ensemble classification, a patch-ensemble classification method is designed, which utilizes the misclassified samples to train patch classifiers for increasing the diversity of base classifiers. The experiments’ results indicate that the designed method has a certain potential for the performance of multi-class imbalanced classification.

EBRB cascade classifier for imbalanced data via rule weight updating

In recent years, data imbalance in the conventional classification problem has raised great interest in the industry. However, concerning the rule-based systems, this problem has been rarely investigated. We propose a Belief Rule-Based (BRB) reasoning model based on the evidential reasoning algorithm to fill in the gap by solving the problem caused by imbalanced data with regards to the rule-based systems. It utilizes the data-driven characteristics of Extended Belief Rule-Based (EBRB) and incorporates the data class rebalancing technique with the ensemble learning method of EBRB. In order to increase the number of rules that can promote the correct classification of minority classes in EBRB, we apply an adaptive data resampling strategy to expand the proportion of minority classes. On this basis, the data is divided into several EBRB base learners with different classification characteristics. During the iterative process, the rule weight of the base learner is dynamically adjusted according to the correct and incorrect classification, such that the classifier can pay more attention to the minority classes and reduce the bias of classification. Finally, the optimized EBRB base learner is selected to build an ensemble classifier with excellent classification performance. Experiments on 27 binary class and 12 multi-class imbalanced datasets show that our approach significantly improves F-value and MACC compared with other imbalanced classification algorithms. By building a cascaded EBRB model, the activation weight is taken as the basis of model evolution, the application limitations of the EBRB model in the ensemble learning are solved innovatively, and the classification performance of the EBRB model for imbalanced data is improved significantly.

A Consolidated Decision Tree-Based Intrusion Detection System for Binary and Multiclass Imbalanced Datasets

The widespread acceptance and increase of the Internet and mobile technologies have revolutionized our existence. On the other hand, the world is witnessing and suffering due to technologically aided crime methods. These threats, including but not limited to hacking and intrusions and are the main concern for security experts. Nevertheless, the challenges facing effective intrusion detection methods continue closely associated with the researcher’s interests. This paper’s main contribution is to present a host-based intrusion detection system using a C4.5-based detector on top of the popular Consolidated Tree Construction (CTC) algorithm, which works efficiently in the presence of class-imbalanced data. An improved version of the random sampling mechanism called Supervised Relative Random Sampling (SRRS) has been proposed to generate a balanced sample from a high-class imbalanced dataset at the detector’s pre-processing stage. Moreover, an improved multi-class feature selection mechanism has been designed and developed as a filter component to generate the IDS datasets’ ideal outstanding features for efficient intrusion detection. The proposed IDS has been validated with state-of-the-art intrusion detection systems. The results show an accuracy of 99.96% and 99.95%, considering the NSL-KDD dataset and the CICIDS2017 dataset using 34 features.