Multi-class Imbalance Problem Research Articles

Multi-class imbalance problem: A multi-objective solution

Multi-class imbalance problems are frequently encountered in real-world applications of machine learning. They have fundamentally complex trade-offs between classes. Existing literature tends to use a predetermined rebalancing strategy and mainly focuses on overall performance measures. However, in many real-world problems, the true level of imbalance and the relative importance between classes are unknown, making it difficult to predetermine the rebalancing strategy and the evaluation criterion. In this paper, we explicitly consider the between-class trade-off issue in the multi-class imbalance problem. We consider all the classes to be important and find a set of optimal trade-offs for the decision-maker to choose from. To reduce the computational cost of this process and make it a practical method, we seek the help of selective ensemble and multiple undersampling rates, and propose the Multi-class Multi-objective Selective Ensemble (MMSE) framework. We further equip the objective modeling with margins to reduce the number of objectives when the task has many classes. Experimental results show that our proposed methods successfully obtain diverse and highly competitive solutions within an acceptable running time.

Multi-Class Imbalance Problem: A Multi-Objective Solution

Multi-class imbalance problems are frequently encountered in real-world applications of machine learning. They have fundamentally complex trade-offs between classes. Existing literature tends to use a predetermined rebalancing strategy and mostly focuses on overall performance measures. However, in many real-world problems, the true level of imbalance and the relative importance between classes are unknown, making it difficult to predetermine the rebalancing strategy and the evaluation criterion. In this paper, we explicitly consider the between-class trade-off issue in the multi-class imbalance problem. We consider all the classes to be important and find a set of optimal trade-offs for the decision-maker to choose from. To reduce the computational cost of this process and make it a practical method, we seek the help of selective ensemble and multiple undersampling rates, and propose the Multi-class Multi-objective Selective Ensemble (MMSE) framework. We further equip the objective modeling with margins to reduce the number of objectives when the task has many classes. Experimental results show that our proposed methods successfully obtain diverse and highly competitive solutions within an acceptable running time.

STO-CVAE: state transition-oriented conditional variational autoencoder for data augmentation in disability classification

The class imbalance problem occurs when there is an unequal distribution of classes in a dataset and is a significant issue in various artificial intelligence applications. This study focuses on the severe multiclass imbalance problem of human activity recognition in rehabilitation exercises for people with disabilities. To overcome this problem, we present a novel human action-centric augmentation method for human skeleton-based pose estimation. This study proposes the state transition-oriented conditional variational autoencoder (STO-CVAE) to capture action patterns in repeated exercises. The proposed approach generates action samples by capturing temporal information of human skeletons to improve the identification of minority disability classes. We conducted experimental studies with a real-world dataset gathered from rehabilitation exercises and confirmed the superiority and effectiveness of the proposed method. Specifically, all investigated classifiers (i.e., random forest, support vector machine, extreme gradient boosting, light gradient boosting machine, and TabNet) trained with the proposed augmentation method outperformed the models trained without augmentation in terms of the F1-score and accuracy, with F1-score showing the most improvement. Overall, the prediction accuracy of most classes was improved; in particular, the prediction accuracy of the minority classes was greatly improved. Hence, the proposed STO-CVAE can be used to improve the accuracy of disability classification in the field of physical medicine and rehabilitation and to provide suitable personal training and rehabilitation exercise programs.

To Combat Multiclass Imbalanced Problems by Aggregating Evolutionary Hierarchical Classifiers.

Real-world datasets are often imbalanced, posing frequent challenges to canonical machine learning algorithms that assume a balanced class distribution. Moreover, the imbalance problem becomes more complicated when the dataset is multiclass. Although many approaches have been presented for imbalanced learning (IL), research on the multiclass imbalanced problem is relatively limited and deficient. To alleviate these issues, we propose a forest of evolutionary hierarchical classifiers (FEHC) method for multiclass IL (MCIL). FEHC can be seen as a classifier fusion framework with a forest structure, and it aggregates several evolutionary hierarchical multiclassifiers (EHMCs) to reduce generalization error. Specifically, a multichromosome genetic algorithm (MCGA) is designed to simultaneously select (sub)optimal features, classifiers, and hierarchical structures when generating these EHMCs. The MCGA adopts a dynamic weighting module to learn difficult classes and promote the diversity of FEHC. We also present the "stratified underbagging" (SUB) strategy to address class imbalance and the "soft tree traversal" (STT) strategy to make FEHC converge faster and better. We thoroughly evaluate the proposed algorithm using 14 multiclass imbalanced datasets with various properties. Compared with popular and state-of-the-art approaches, FEHC obtains better performance under different evaluation metrics. Codes have been made publicly available on GitHub.https://github.com/CUHKSZ-NING/FEHCClassifier.

A hybrid artificial intelligence algorithm for fault diagnosis of hot rolled strip crown imbalance

In the production process of hot continuous rolling, due to the imbalance between the number of normal cases and fault cases, the traditional supervised learning methods are often unable to deal with them efficiently. In order to address this problem, a new Hybrid Artificial Intelligence Algorithm is proposed, which is based on Light Gradient Boosting Machine (LightGBM) and combines resampling technique with Support Vector Machine (SVM) to deal with multi-class imbalance problems. Specifically, Support Vector Machines-Synthetic Minority Oversampling Technique (SVM-SMOTE) is used to increase the training data, so as to change the distribution of training data and improve the accuracy of the model, and the Particle Swarm Optimization (PSO) algorithm is used to find the hyper parameters of the model to determine the optimal hyper parameters combination. At the same time, the method of SHapley Additive exPlanation (SHAP) based on game theory is used to reveal the factors affecting the convexity of strip steel plate in hot rolling process and its interpretability. In check to see the validity of the model, experiments were carried out on UCI data set and hot rolling production data set, and multi-model comparison was carried out. The values of Geometric mean (G-mean), macroscopic F1 fraction (F1-Macro) and Matthews correlation coefficient (MCC) tested on the hot rolling data set are 0.992, 0.992 and 0.987 respectively. The model has a good effect. The results show that the model is significantly better than traditional models in solving the problem of strip convexity diagnosis during hot rolling process.

A random forest approach to quality-checking automatic snow-depth sensor measurements

Abstract. State-of-the-art snow sensing technologies currently provide an unprecedented amount of data from both remote sensing and ground sensors, but their assimilation into dynamic models is bounded to data quality, which is often low – especially in mountain, high-elevation, and unattended regions where snow is the predominant land-cover feature. To maximize the value of snow-depth measurements, we developed a random forest classifier to automatize the quality assurance and quality control (QA/QC) procedure of near-surface snow-depth measurements collected through ultrasonic sensors, with particular reference to the differentiation of snow cover from grass or bare-ground data and to the detection of random errors (e.g., spikes). The model was trained and validated using a split-sample approach of an already manually classified dataset of 18 years of data from 43 sensors in Aosta Valley (northwestern Italian Alps) and then further validated using 3 years of data from 27 stations across the rest of Italy (with no further training or tuning). The F1 score was used as scoring metric, it being the most suited to describe the performances of a model in the case of a multiclass imbalanced classification problem. The model proved to be both robust and reliable in the classification of snow cover vs. grass/bare ground in Aosta Valley (F1 values above 90 %) yet less reliable in rare random-error detection, mostly due to the dataset imbalance (samples distribution: 46.46 % snow, 49.21 % grass/bare ground, 4.34 % error). No clear correlation with snow-season climatology was found in the training dataset, which further suggests the robustness of our approach. The application across the rest of Italy yielded F1 scores on the order of 90 % for snow and grass/bare ground, thus confirming results from the testing region and corroborating model robustness and reliability, with again a less skillful classification of random errors (values below 5 %). This machine learning algorithm of data quality assessment will provide more reliable snow data, enhancing their use in snow models.

A partition-based problem transformation algorithm for classifying imbalanced multi-label data

Multi-label learning has garnered much research interest due to its wide range of real-world applications. Many multi-label learning methods have been proposed; however, few have addressed the class imbalance problem existing in multi-label data. Even though some studies have taken this issue into account, most of them have ignored the label correlations or only considered random correlations between them. In this study, we propose a novel partition-based imbalanced multi-label learning algorithm, named Multi-label Learning based on Hierarchical Clustering (MLHC), to tackle this problem. MLHC first carries out hierarchical clustering on the original label space to divide it into several disconnected subspaces, each of which contains several labels that are strongly correlated with each other. Then, for each label subspace, we use the problem transformation strategy to convert it into a multi-class problem by binary coding. Any multi-class imbalance learning algorithm can be applied to the transformed multi-class data. Finally, the classification results will be decoded to retrieve the corresponding label subspace, and all label subspace results are combined to show the predicted label vector in the original label space. We conducted experiments not only on thirteen benchmark multi-label datasets but also carried out them on XJTU-SY which is a multi-label engineering application dataset, and the results indicated that our proposed MLHC learning algorithm outperforms several state-of-the-art class imbalance multi-label learning algorithms, demonstrating the effectiveness and necessity of discovering label correlations and transforming the original imbalanced multi-label learning problem into multiple strongly correlated multi-class imbalanced learning problems.

Transformer Dissolved Gas Analysis for Highly-Imbalanced Dataset Using Multiclass Sequential Ensembled ELM

Dissolved gas analysis (DGA) has been a critical technique for transformer diagnosis. DGA is a typical multi-class imbalance problem where most of the samples correspond to healthy state transformers or units. Though numerous works have been carried out on this issue, the diagnosis accuracy is still unsatisfactory when the status of health and multiple faults are considered. Multi-class imbalance problem is also a tough task from the view of algorithm development. Previous works underestimate this issue in some sakes such as lacking investigation of the highly imbalanced dataset, and lacking consideration of health data. This paper presents a comprehensive study of the mentioned issues. A novel algorithm called sequential ensembled extreme learning machine (SE-ELM) is proposed. SE-ELM adopts a novel multi-class undersampling strategy followed by a sequentially updated ensemble, which achieves both accuracy and efficiency. The proposed method is validated on both an open IEC dataset and a highly imbalanced private dataset. The comparison with popular algorithms proves the efficiency of SE-ELM.

Extreme Gradient Boosting Algorithm to Improve Machine Learning Model Performance on Multiclass Imbalanced Dataset

Unbalanced conditions in the dataset often become a real-world problem, especially in machine learning. Class imbalance in the dataset is a condition where the number of minority classes is much smaller than the majority class, or the number is insufficient. Machine learning models tend to recognize patterns in the majority class more than in the minority class. This problem is one of the most critical challenges in machine learning research, so several methods have been developed to overcome it. However, most of these methods only focus on binary datasets, so few methods still focus on multiclass datasets. Handling unbalanced multiclass is more complex than handling unbalanced binary because it involves more classes than binary class datasets. With these problems, we need an algorithm with features that can support adjustments to the difficulties that arise in multiclass unbalanced datasets. One of the algorithms that have features for adjustment is the ensemble algorithm, namely Xtreme Gradient Boosting. Based on the research, our proposed method with Xtreme Gradient Boosting showed better results than the other classification and ensemble algorithms on eight datasets with five evaluation metrics indicators such as balanced accuracy, the geometric-mean, multiclass area under the curve, true positive rate, and true negative rate. In future research, we suggest combining methods at the data level and Xtreme Gradient Boosting. With the performance increase in Xtreme Gradient Boosting, it can be a solution and reference in the case of handling multiclass imbalanced problems. Besides, we also recommended testing with datasets in the form of categorical and continuous data.

Application of deep reinforcement learning for spike sorting under multi-class imbalance

Spike sorting is the basis for analyzing spike firing patterns encoded in high-dimensional information spaces. With the fact that high-density microelectrode arrays record multiple neurons simultaneously, the data collected often suffers from two problems: a few overlapping spikes and different neuronal firing rates, which both belong to the multi-class imbalance problem. Since deep reinforcement learning (DRL) assign targeted attention to categories through reward functions, we propose ImbSorter to implement spike sorting under multi-class imbalance. We describe spike sorting as a Markov sequence decision and construct a dynamic reward function (DRF) to improve the sensitivity of the agent to minor classes based on the inter-class imbalance ratios. The agent is eventually guided by the optimal strategy to classify spikes. We consider the Wave_Clus dataset, which contains overlapping spikes and diverse noise levels, and the macaque dataset, which has a multi-scale imbalance. ImbSorter is compared with classical DRL architectures, traditional machine learning algorithms, and advanced overlapping spike sorting techniques on these two above datasets. ImbSorter obtained improved results on the Macro_F1. The results show ImbSorter has a promising ability to resist overlapping and noise interference. It has high stability and promising performance in processing spikes with different degrees of skewed distribution.

Hybrid Approach with Distance Feature for Multi-Class Imbalanced Datasets

The multi-class imbalance problem has a higher level of complexity when compared to the binary class problem. The difficulty is due to the large number of classes that will present challenges related to overlapping between classes. Many approaches have been proposed to deal with these multi-class problems. One is a hybrid approach combining a data-level approach and an algorithm-level approach. This approach is done by the ensemble on the classifier and also oversampling on the minority class. SMOTE is an oversampling method that provides good performance, but this method is necessary to determine the best sample used in the interpolation process to generate new samples. The need for determining the best sample is related to the overlap between classes that always accompanies the multi-class imbalance problem. The existence of overlap requires efforts to determine the safe region to synthesize the sample in the oversampling process in SMOTE. The safe region is considered the best for synthesizing samples due to the lower tendency of overlapping. It can be done by constructing distance features to determine the safe region. The sample with the best distance and the lowest imbalance ratio will be selected as a sample in the over-sampling process with SMOTE. The main contribution of this research is the proposed method of Hybrid Approach with Distance Feature so that it can determine safe samples, with the main advantage being in addition to handling multi-class imbalances, it is also better for handling overlapping. The results of this study will be compared with Multiple Random Balance (MultiRandBal) which performs a random oversampling process. The results showed that the Augmented R-Value, Class Average Accuracy, Class Balance Accuracy, and Hamming Loss obtained in this method was better than the random oversampling process. These results also show that the Hybrid Approach with Distance Feature provides better results in handling multi-class imbalances when compared to MultiRandBal.

An Experimental Analysis of Drift Detection Methods on Multi-Class Imbalanced Data Streams

The performance of machine learning models diminishes while predicting the Remaining Useful Life (RUL) of the equipment or fault prediction due to the issue of concept drift. This issue is aggravated when the problem setting comprises multi-class imbalanced data. The existing drift detection methods are designed to detect certain drifts in specific scenarios. For example, the drift detector designed for binary class data may not produce satisfactory results for applications that generate multi-class data. Similarly, the drift detection method designed for the detection of sudden drift may struggle with detecting incremental drift. Therefore, in this experimental investigation, we seek to investigate the performance of the existing drift detection methods on multi-class imbalanced data streams with different drift types. For this reason, this study simulated the streams with various forms of concept drift and the multi-class imbalance problem to test the existing drift detection methods. The findings of current study will aid in the selection of drift detection methods for use in developing solutions for real-time industrial applications that encounter similar issues. The results revealed that among the compared methods, DDM produced the best average F1 score. The results also indicate that the multi-class imbalance causes the false alarm rate to increase for most of the drift detection methods.

Traffic Prediction System Using IoT Cluster Based Evolutionary Under Sampling Approach

Road traffic is increasing nowadays due to the rapid growth of vehicle usage, which necessitates highly accurate traffic prediction systems. The traffic data is time-series data which identifies different pattern during low, moderate and high traffic time. Moving weighted average model that captures this phenomenon by assigning different weights for the different traffic classes to calculate the threshold values. This paper addresses the class imbalance in the traffic dataset due to which has the high impact on precision. The class imbalance problem can be handled by various data-level and algorithm-level techniques. Hybrid methods show better results while handling skewed class distribution in binary class problems. But the multiclass imbalance is a rarely focused research area. In this work, Cluster Based Evolutionary Undersampling with Correlation Relation (CBEUS_CR) is proposed to handle the multiclass imbalance problem of the traffic prediction system. The proposed system uses multiple regression as a fitness function on traffic parameters of an evolutionary algorithm in the majority classes on Apache spark environment. As a result, the sensitivity, and specificity of traffic prediction is improved and achieves 97% accuracy as it balances the number of instances between majority and minority classes.

Novel hybrid classification model for multi-class imbalanced lithology dataset

Deep learning methods and applications assist geologists in predicting and identifying lithologies in different surveys, hence lowering operational costs and uptime. This allows accurate data analysis and completion of scientific research on data obtained at geologically different places. This study used 4 lithologies datasets with high dimensionality and multiclass imbalance problems for analysis and classification. The imbalance of data classification is one of the most important problems facing current data analysis. Data imbalance can considerably influence classification performance, especially when dealing with other difficulty variables such as the presence of overlapping class distributions. This impact is especially obvious in multi-class conditions when mutual imbalance relations across classes complicate matters even further. Moreover, the problem of high dimensionality can lead to increased computing complexity and overfitting, and thus these issues can affect classification performance. To overcome these problems, we developed a new hybrid deep learning multi-class imbalanced learning method that combines Synthetic Minority Oversampling (SMOTE) to resample the data, and Recursive Feature Elimination (RFE) to identify the most useful predictive features. Finally, we believe that our developments can help improve geology research by providing accurate classification and rapid answers about interpreting data obtained in various study areas.

A Parallel Cross Convolutional Recurrent Neural Network for Automatic Imbalanced ECG Arrhythmia Detection with Continuous Wavelet Transform.

Automatic detection of arrhythmia using electrocardiogram (ECG) and deep learning (DL) is very important to reduce the global death rate from cardiovascular diseases (CVD). Previous studies on automatic arrhythmia detection relied largely on various ECG features and have achieved considerable classification accuracy using DL-based models. However, most previous research has ignored multi-class imbalanced problems in ECG arrhythmia detection. Therefore, it remains a challenge to improve the classification performance of the DL-based models. This paper proposes a novel parallel cross convolutional recurrent neural network in order to improve the arrhythmia detection performance of imbalanced ECG signals. The proposed model incorporates a recurrent neural network and a two-dimensional (2D) convolutional neural network (CNN) and can effectively learn temporal characteristics and rich spatial information of raw ECG signals. Continuous wavelet transform (CWT) is used to transform the ECG signals into a 2D scalogram composed of time–frequency components, and subsequently, the 2D-CNN can learn spatial information from the 2D scalogram. The proposed model is not only efficient in learning features with imbalanced samples but can also significantly improve model convergence with higher accuracy. The overall performance of our proposed model is evaluated based on the MIT-BIH arrhythmia dataset. Detailed analysis of evaluation metrics reveals that the proposed model is very effective in arrhythmia detection and significantly better than the existing hierarchical network models.

Generation of Controlled Synthetic Samples and Impact of Hyper-Tuning Parameters to Effectively Classify the Complex Structure of Overlapping Region

The classification of imbalanced and overlapping data has provided customary insight over the last decade, as most real-world applications comprise multiple classes with an imbalanced distribution of samples. Samples from different classes overlap near class boundaries, creating a complex structure for the underlying classifier. Due to the imbalanced distribution of samples, the underlying classifier favors samples from the majority class and ignores samples representing the least minority class. The imbalanced nature of the data—resulting in overlapping regions—greatly affects the learning of various machine learning classifiers, as most machine learning classifiers are designed to handle balanced datasets and perform poorly when applied to imbalanced data. To improve learning on multi-class problems, more expertise is required in both traditional classifiers and problem domain datasets. Some experimentation and knowledge of hyper-tuning the parameters and parameters of the classifier under consideration are required. Several techniques for learning from multi-class problems have been reported in the literature, such as sampling techniques, algorithm adaptation methods, transformation methods, hybrid methods, and ensemble techniques. In the current research work, we first analyzed the learning behavior of state-of-the-art ensemble and non-ensemble classifiers on imbalanced and overlapping multi-class data. After analysis, we used grid search techniques to optimize key parameters (by hyper-tuning) of ensemble and non-ensemble classifiers to determine the optimal set of parameters to enhance the learning from a multi-class imbalanced classification problem, performed on 15 public datasets. After hyper-tuning, 20% of the dataset samples are synthetically generated to add to the majority class of each respective dataset to make it more overlapped (complex structure). After the synthetic sample’s addition, the hyper-tuned ensemble and non-ensemble classifiers are tested over that complex structure. This paper also includes a brief description of tuned parameters and their effects on imbalanced data, followed by a detailed comparison of ensemble and non-ensemble classifiers with the default and tuned parameters for both original and synthetically overlapped datasets. We believe that the underlying paper is the first kind of effort in this domain, which will furnish various research aspects to with a greater focus on the parameters of the classifier in the field of learning from imbalanced data problems using machine-learning algorithms.

A Novel Double Ensemble Algorithm for the Classification of Multi-Class Imbalanced Hyperspectral Data

The class imbalance problem has been reported to exist in remote sensing and hinders the classification performance of many machine learning algorithms. Several technologies, such as data sampling methods, feature selection-based methods, and ensemble-based methods, have been proposed to solve the class imbalance problem. However, these methods suffer from the loss of useful information or from artificial noise, or result in overfitting. A novel double ensemble algorithm is proposed to deal with the multi-class imbalance problem of the hyperspectral image in this paper. This method first computes the feature importance values of the hyperspectral data via an ensemble model, then produces several balanced data sets based on oversampling and builds a number of classifiers. Finally, the classification results of these diversity classifiers are combined according to a specific ensemble rule. In the experiment, different data-handling methods and classification methods including random undersampling (RUS), random oversampling (ROS), Adaboost, Bagging, and random forest are compared with the proposed double random forest method. The experimental results on three imbalanced hyperspectral data sets demonstrate the effectiveness of the proposed algorithm.

Double-kernel based class-specific broad learning system for multiclass imbalance learning

Imbalance learning has gained more and more attention from researchers. Most of the efforts so far have focused on binary imbalance problems, while there are numerous unresolved multiclass imbalance problems in real-world scenarios. The diversity of data distribution and the poor performance of traditional multiclass classification algorithms present significant challenges for classifying multiclass imbalanced data. This paper proposes a double kernel-based class-specific broad learning system (DKCSBLS) for multi-class imbalance learning. Class-specific penalty coefficients are incorporated into the model to increase the focus on minority classes. Moreover, double kernel mapping mechanism is designed to extract more robust features. Extensive experiments on various real-world datasets demonstrate the superiority of our proposed approach.

What makes multi-class imbalanced problems difficult? An experimental study

Multi-class imbalanced classification is more difficult and less frequently studied than its binary counterpart. Moreover, research on the causes of the difficulty of multi-class imbalanced data is quite limited and insufficient. Therefore, we experimentally study the impact of various multi-class imbalanced difficulty factors on the performance of three popular classifiers. The results demonstrated a strong influence of the class overlapping with the extent of its impact related to the types of overlapped classes. In particular, overlapping between minority and majority classes was more difficult than the others. The type of the class size configuration turned out to be another important factor, highlighting the special role of the configurations with classes of intermediate sizes. The obtained results could support studying the nature of the multi-class imbalanced data as well as the development of new methods for improving classifiers.

Performance Analysis of Two-Stage Iterative Ensemble Method over Random Oversampling Methods on Multiclass Imbalanced Datasets

Data imbalance occurring among multiclass datasets is very common in real-world applications. Existing studies reveal that various attempts were made in the past to overcome this multiclass imbalance problem, which is a severe issue related to the typical supervised machine learning methods such as classification and regression. But, still there exists a need to handle the imbalance problem efficiently as the datasets include both safe and unsafe minority samples. Most of the widely used oversampling techniques like SMOTE and its variants face challenges in replicating or generating the new data instances for balancing them across multiple classes, particularly when the imbalance is high and the number of rare samples count is too minimal thus leading the classifier to misclassify the data instances. To lessen this problem, we proposed a new data balancing method namely a two-stage iterative ensemble method to tackle the imbalance in multiclass environment. The proposed approach focuses on the rare minority sample’s influence on learning from imbalanced datasets and the main idea of the proposed approach is to balance the data without any change in class distribution before it gets trained by the learner such that it improves the learner’s learning process. Also, the proposed approach is compared against two widely used oversampling techniques and the results reveals that the proposed approach shows a much significant improvement in the learning process among the multiclass imbalanced data.