Multi-class Imbalance Research Articles

Detection of mass imbalance in the rotor of wind turbines using Support Vector Machine

Condition monitoring systems (CMS) are essential to reduce costs in the wind energy sector. This paper proposes a method based on Support Vector Machine (SVM) to detect rotor mass imbalance for a multi-class imbalance problem, using the estimated speed as an input variable, obtained through a combination of electrical quantities (currents and voltages). Moreover, it is sought to obtain the magnitude of the rotor mass imbalance. With the aid of statistical tools, intermediate classes can be estimated, other than the ones proposed for the SVM. Besides, if the azimuth position is provided, the angular position of the mass imbalance can be also obtained. A 1.5 MW three-bladed wind turbine model with a permanent magnet synchronous generator, was considered, and a database was built numerically using the software Turbsim, FAST, and Simulink. From the database, the Power Spectral Density (PSD) technique was used to transform the input data from the time to the frequency domain. Then, the SVM algorithm and statistical analysis were used to classify the magnitude and the angular position of the imbalance. Different scenarios of mass imbalance were tested under different wind speeds and turbulence intensities. The results demonstrate the satisfactory performance of the proposed method.

Smart Approach for Glioma Segmentation in Magnetic Resonance Imaging using Modified Convolutional Network Architecture (U-NET)

Segmentation of a brain tumor from magnetic resonance multimodal images is a challenging task in the field of medical imaging. The vast diversity in potential target regions, appearance and multifarious intensity threshold levels of various tumor types are few of the major factors that affect segmentation results. An accurate diagnosis and its treatment demand strict delineation of the tumor affected tissues. Herein, we focus on a smart, automated, and robust segmentation approach for brain tumor using a modified 3D U-Net architecture. The pre-operative multimodal 3D-MRI scans of High-Grade Glioma (HGG) and Low-Grade Glioma (LGG) are used as data. Our proposed approach solves the problem of memory and system resource constraints by robustly applying dense network training on image patches of 3D volumes. It improves the border region artifact detection by applying convolutions at an appropriate phase in the proposed neural network. Multi-class imbalance data are handled by using Categorical Cross Entropy (CCE) loss developed by combining the Weighted Cross Entropy (WCE) with Weighted Multi-class Dice Loss (WMDL) functions, which enables the network to perform smart segmentation of the smaller tumorous regions. The proposed approach is tested and evaluated for the challenge datasets of multimodal MRI volumes of tumor patients. Experiments are performed to compute the average dice scores on BraTS-2019 and BraTS-2020 datasets for the whole tumor region.

Class-discriminative focal loss for extreme imbalanced multiclass object detection towards autonomous driving

Currently, modern object detection algorithms still suffer the imbalance problems especially the foreground–background and foreground–foreground class imbalance. Existing methods generally adopt re-sampling based on the class frequency or re-weighting based on the category prediction probability, such as focal loss, proposed to rebalance the loss assigned to easy negative examples and hard positive examples for single-stage detectors. However, there are still two critical issues unresolved. In practical applications, such as autonomous driving, the class imbalance will become more extreme due to the increased detection field and target distribution characteristics, needing a more effective way to balance the foreground–background class imbalance. Besides, existing methods typically employ the sigmoid or softmax entropy loss for classification task, which we believe is not capable to realize the foreground–foreground class balance. In this paper, we propose a new form of focal loss by re-designing the re-weighting scheme that can calculate the weight according to the probability as well as widen the weight difference of the examples. Besides, we introduce the extended focal loss to multi-class classification task by reformulating the standard softmax cross-entropy loss for better utilizing the discriminant difference of foreground categories, thereby yielding a class-discriminative focal loss. Comprehensive experiments are conducted on the KITTI and BDD dataset, respectively. The results show that our approach can easily surpass focal loss with no more training and inference time cost. Besides, when trained with the proposed loss function, current state-of-the-art object detectors no matter in one-stage or two-stage paradigms can achieve significant performance gains.

A comprehensive active learning method for multiclass imbalanced data streams with concept drift

A challenge to many real-world applications is multiclass imbalance with concept drift. In this paper, we propose a comprehensive active learning method for multiclass imbalanced streaming data with concept drift (CALMID). First, we design a comprehensive online active learning framework that includes an ensemble classifier, a drift detector, a label sliding window, sample sliding windows and an initialization training sample sequence. Next, a variable threshold uncertainty strategy based on an asymmetric margin threshold matrix is designed to comprehensively address the problem that a given class can simultaneously be a majority to a given subset of classes while also being a minority to others. Last but not least, we design a novel sample weight formula that comprehensively considers the class imbalance ratio of the sample’s category and the prediction difficulty. On 10 multiclass synthetic streams with different imbalance ratios and concept drifts, and on 5 real-world imbalanced streams with 7 to 55 classes and unknown drifts, the experimental results demonstrate that the proposed CALMID is more effective and efficient than several state-of-the-art learning algorithms.

Automatic cardiac cine MRI segmentation and heart disease classification

Cardiac cine magnetic resonance imaging (MRI) continues to be recognized as an established modality for non-invasive assessment of the function and structure of the cardiovascular system. Making full use of fully convolutional neural networks CNNs ability to operate pixel-wise classification, cine MRI sequences can be segmented and volumetric features of three key heart structures are computed for disease prediction. The three key heart structures are the left ventricle cavity, right ventricle cavity and the left ventricle myocardium. In this paper, we suggest an automated pipeline for both cardiac segmentation and diagnosis. The study was conducted on a dataset of 150 patients from Dijon Hospital in the context of the post-2017 Medical Image Computing and Computer Assisted Intervention MICCAI, Automated Cardiac Diagnosis Challenge (ACDC). The challenge consists in two phases: (i) a segmentation contest, where performance is evaluated on dice overlap coefficient and Hausdorff distance metrics, and a (ii) diagnosis contest for heart disease classification. For this aim, we propose the use of a deep learning based network for segmentation of the three key cardiac structures within short-axis cine MRI sequences and a classifier ensemble for heart disease classification. The deep learning segmentation network is a UNet fully convolutional neural network variant with fewer trainable parameters. The classifier ensemble consists in combining three classifiers, namely a multilayer perceptron, a random forest and a support vector machine. Before feeding the segmentation network, a preliminary step consists in localizing heart region and cropping input images to a restricted region of interest (ROI). This is achieved by a signal processing based approach and aims at reducing multi-class imbalance and computational load. We achieved nearly state of the art accuracy performances for both the segmentation and disease classification challenges. Reporting a mean dice overlap coefficient of 0.92 for the three cardiac structures segmentation, along with good limits of agreement for the various derived clinical indices, leading to an accuracy of 0.92 for the disease classification on unseen data.

Enhanced-Random-Feature-Subspace-Based Ensemble CNN for the Imbalanced Hyperspectral Image Classification

Hyperspectral image (HSI) classification often faces the problem of multiclass imbalance, which is considered to be one of the major challenges in the field of remote sensing. In recent years, deep learning has been successfully applied to the HSI classification, a convolutional neural network (CNN) is one of the most representative of them. However, it is difficult to effectively improve the accuracy of minority classes under the problem of multiclass imbalance. In addition, ensemble learning has been successfully applied to solve multiclass imbalance, such as random forest (RF) This article proposes a novel enhanced-random-feature-subspace-based ensemble CNN algorithm for the multiclass imbalanced problem. The main idea is to perform random oversampling of training samples and multiple data enhancements based on random feature subspace, and then, construct an ensemble learning model combining random feature selection and CNN to the HSI classification. Experimental results on three public hyperspectral datasets show that the performance of the proposed method is better than the traditional CNN, RF, and deep learning ensemble methods.

Effects of Class Imbalance Using Machine Learning Algorithms

Class imbalance is the major hurdle for machine learning-based systems. Data set is the backbone of machine learning and must be studied to handle the class imbalance. The purpose of this paper is to investigate the effect of class imbalance on the data sets. The proposed methodology determines the model accuracy for class distribution. To find possible solutions, the behaviour of an imbalanced data set was investigated. The study considers two case studies with data set divided balanced to unbalanced class distribution. Testing of the data set with trained and test data was carried out for standard machine learning algorithms. Model accuracy for class distribution was measured with the training data set. Further, the built model was tested with individual binary class. Results show that, for the improvement of the system performance, it is essential to work on class imbalance problems. The study concludes that the system produces biased results due to the majority class. In the future, the multiclass imbalance problem can be studied using advanced algorithms.

ENSEMBLE META CLASSIFIER WITH SAMPLING AND FEATURE SELECTION FOR DATA WITH IMBALANCE MULTICLASS PROBLEM

Ensemble learning by combining several single classifiers or another ensemble classifier is one of the procedures to solve the imbalance problem in multiclass data. However, this approach still faces the question of how the ensemble methods obtain their higher performance. In this paper, an investigation was carried out on the design of the meta classifier ensemble with sampling and feature selection for multiclass imbalanced data. The specific objectives were: 1) to improve the ensemble classifier through data-level approach (sampling and feature selection); 2) to perform experiments on sampling, feature selection, and ensemble classifier model; and 3 ) to evaluate t he performance of the ensemble classifier. To fulfil the objectives, a preliminary data collection of Malaysian plants’ leaf images was prepared and experimented, and the results were compared. The ensemble design was also tested with three other high imbalance ratio benchmark data. It was found that the design using sampling, feature selection, and ensemble classifier method via AdaboostM1 with random forest (also an ensemble classifier) provided improved performance throughout the investigation. The result of this study is important to the on-going problem of multiclass imbalance where specific structure and its performance can be improved in terms of processing time and accuracy.

A modified real-value negative selection detector-based oversampling approach for multiclass imbalance problems

A skewed distribution poses a major challenge in multiclass imbalanced problems and has attracted growing interest in the engineering and research communities. Conventional classifiers pose limitations in handling multiclass imbalanced data sets since they were originally designed to handle a balanced distribution. This paper proposes a modified real-value negative selection detector-based oversampling approach for the multiclass imbalance problem. Different from previous works, we have modified and introduced the traditional real-value negative selection algorithm to address the issue of the multiclass imbalance problem. First, the modified real-value negative selection technique is used to create detectors for each minority class. Then, a supervision mechanism is designed using these detectors to prevent overgeneralization. Furthermore, the method of selection weight based on crowding density and detectors is devised with the aim of reducing the within-class imbalance for each minority class. Finally, simulations were conducted on the public real-world multiclass imbalanced data sets from Knowledge Extraction based on Evolutionary Learning (KEEL) and the University of California Irvine (UCI). The experimental and statistical results have demonstrated that the proposed algorithm obtains better performance compared with seven well-known oversampling algorithms in terms of five metrics (precision, recall, F-measure, Multiclass G-mean (MG), and Multiclass Area Under the ROC (MAUC)) and four types of classifiers.

Ensemble Method to classify multi class with concept drift

Real world scenarios is expected to have more number of classes, evolution of new class, change in concept, different distribution of data etc., This paper focus on the joint problem of multi class imbalance problem and different types of concept drift detection. The proposed work Enhanced Concept drift based Resampling ensemble (ECDRE) algorithm addresses the combined issues of Multi Class Imbalance problem with different types of Concept Drifts using the Ensemble Relearning Model. In this framework, EMUOB and EMOOB algorithm helps to re-balance the multi class distribution of the latest block. Window based strategy is used to classify the multi class imbalance dataset with different types of drifts. A group of classifiers form an ensemble and members belonging to the ensembles are updated frequently, where weights are calculated to emphasis misclassification cost. Meanwhile, ECDRE is swiftly adaptable to new conditions. Observed studies proof that ECDRE algorithm is more effective for learning non-stationary environment which suffers from multi class imbalance with different types of concept drift. CDRE algorithm was implemented to deal with multi-class problem and concept drift, but it is not much efficient to handle all types of concept drift. Even in situations of Data Streams which suffer from drifts such as real, virtual and recurring types, ECDRE works well and shows overall better results of above 80% accuracy.

Towards Class-Imbalance Aware Multi-Label Learning.

Multi-label learning deals with training examples each represented by a single instance while associated with multiple class labels. Due to the exponential number of possible label sets to be considered by the predictive model, it is commonly assumed that label correlations should be well exploited to design an effective multi-label learning approach. On the other hand, class-imbalance stands as an intrinsic property of multi-label data which significantly affects the generalization performance of the multi-label predictive model. For each class label, the number of training examples with positive labeling assignment is generally much less than those with negative labeling assignment. To deal with the class-imbalance issue for multi-label learning, a simple yet effective class-imbalance aware learning strategy called cross-coupling aggregation (COCOA) is proposed in this article. Specifically, COCOA works by leveraging the exploitation of label correlations as well as the exploration of class-imbalance simultaneously. For each class label, a number of multiclass imbalance learners are induced by randomly coupling with other labels, whose predictions on the unseen instance are aggregated to determine the corresponding labeling relevancy. Extensive experiments on 18 benchmark datasets clearly validate the effectiveness of COCOA against state-of-the-art multi-label learning approaches especially in terms of imbalance-specific evaluation metrics.

Multi-Class Imbalance in Text Classification: A Feature Engineering Approach to Detect Cyberbullying in Twitter

Twitter enables millions of active users to send and read concise messages on the internet every day. Yet some people use Twitter to propagate violent and threatening messages resulting in cyberbullying. Previous research has focused on whether cyberbullying behavior exists or not in a tweet (binary classification). In this research, we developed a model for detecting the severity of cyberbullying in a tweet. The developed model is a feature-based model that uses features from the content of a tweet, to develop a machine learning classifier for classifying the tweets as non-cyberbullied, and low, medium, or high-level cyberbullied tweets. In this study, we introduced pointwise semantic orientation as a new input feature along with utilizing predicted features (gender, age, and personality type) and Twitter API features. Results from experiments with our proposed framework in a multi-class setting are promising both with respect to Kappa (84%), classifier accuracy (93%), and F-measure (92%) metric. Overall, 40% of the classifiers increased performance in comparison with baseline approaches. Our analysis shows that features with the highest odd ratio: for detecting low-level severity include: age group between 19–22 years and users with <1 year of Twitter account activation; for medium-level severity: neuroticism, age group between 23–29 years, and being a Twitter user between one to two years; and for high-level severity: neuroticism and extraversion, and the number of times tweet has been favorited by other users. We believe that this research using a multi-class classification approach provides a step forward in identifying severity at different levels (low, medium, high) when the content of a tweet is classified as cyberbullied. Lastly, the current study only focused on the Twitter platform; other social network platforms can be investigated using the same approach to detect cyberbullying severity patterns.

Data Level Approach for Multiclass Imbalance Financial Data

In the real world, the class imbalance problem is a common issue in which classifier gives more importance to the majority class whereas less importance to the minority class. In class imbalance, imbalance metrics would not be suitable to evaluate the performance of classifiers with error rate or predictive accuracy. One type of imbalance data -handling method is resampling. In this paper, three resampling methods, oversampling, under-sampling and hybrid, methods are used with different approaches for in class imbalance of two different financial data to see the impact of class imbalance ratios on performance measures of nine different classification algorithms. Aiming to achieve better change classification performance, the performance of the classification algorithms, Bayes Net, Navie Bayes, J48, Random Forest Meta-Attribute Selected Classifier, MetaClassification via Regression, Meta-Logitboost, Logistic Regression, and Decision Tree, are measured on two Canadian Banks multiclass imbalance data with the performance measures, Precision, Recall, ROC Area and Kappa Statistic, by using WEKA software. The outcome of these performance measurements compared with three different resampling methods. The results provide us with a clear picture on the overall impact of class imbalance on the classification dataset and they indicate that proposed resampling methods can also be used for in class imbalance problems

A class imbalance-aware review rating prediction using hybrid sampling and ensemble learning

Imbalanced distribution of instances across the classes is a challenging issue when the underlying problem is of type classification. The reason is that classifiers will tend to favor the classes with a large number of instances i.e. instances of minority classes may be identified as instances of majority classes by the classifiers. In recent years, plenty of researches have been done to resolve the class imbalance issue in binary classification problems which resulted in many class imbalance learning techniques for binary classification problems. But, the class imbalance in multi-class classification problems did not draw much attention from the research community. Unlike binary class imbalance learning, multi-class imbalance learning techniques experience more than one majority class and more than one minority class. This paper tries to come up with a multi-class imbalanced learning technique that can overcome the effects of multi-class imbalance problem in review rating prediction tasks. The proposed model handles the multi-class imbalance issue by using the combination of hybrid sampling and ensemble learning techniques. Sampling techniques such as Random Under Sampling (RUS) and Synthetic Minority Over-sampling TEchnique(SMOTE) are jointly used in the proposed model to create balanced training sets for base learners. Also, the proposed model creates a powerful ensemble structure by amalgamating a manually created bagging ensemble and AdaBoost boosting ensembles. Experiments are done using the Amazon product dataset in order to investigate the performance of the proposed model. The experimental results show that the proposed Class Imbalance-Aware Review rating prediction(CIAR) model outperforms almost all the baseline models in-terms of G-mean, F-Score, and ROC_AUC_Score.

Imbalance accuracy metric for model selection in multi-class imbalance classification problems

The overall accuracy, macro precision, macro recall, F-score and class balance accuracy, due to their simplicity and easy interpretation, have been among the most popular metrics to measure the performance of classifiers on multi-class problems. However, on imbalance datasets, some of these metrics can be unfairly influenced by heavier classes. Therefore, it is recommended that they are used as a group and not individually. This strategy can unnecessarily complicate the model selection and evaluation in imbalance datasets. In this paper, we introduce a new metric, imbalance accuracy metric (IAM), that can be used as a solo measure for model evaluation and selection. The IAM is built up on top of the existing metrics, is simple to use, and easy to interpret. This metric is meant to be used as a bottom-line measure aiming to eliminate the need for group metric computation and simplify the model selection.

A novel dynamic ensemble selection classifier for an imbalanced data set: An application for credit risk assessment

Credit risk assessment is usually regarded as an imbalanced classification task solved by static ensemble classifiers. However, the dynamic ensemble selection (DES) strategy that can select different ensemble classifiers for each query sample is rarely used. Deficiency of the existing DES algorithm in dealing with imbalanced data is the major challenge. In this paper, a novel combined DES model is developed for imbalanced learning problems. To handle the imbalanced data sets, the synthetic minority over-sampling technique is initially used to balance a training set before generating a candidate classifier pool; then, the weighting mechanism of DES-MI (multi-class imbalance) is used to highlight the importance of minority instances when evaluating classifier competences. To further ensure the comprehensive evaluation and right selection of the ensemble classifier, the meta-learning framework of META-DES is used to account for multiple criteria, and the two-step selection strategy of DES-KNN (k-nearest neighbours) is employed to perform a trade-off between the competence and diversity of the classifiers. Our experiments on 15 imbalanced data sets from the KEEL repository show that the proposed model improves the performance of seven known and popular DES algorithms in terms of the area under the curve. Moreover, the type I error rate of the proposed method is lower than that of XGBoost and LightGBM in a real P2P loan data set indicating the efficiency of the proposed method for credit risk assessment.

M-AdaBoost-A based ensemble system for network intrusion detection

Network intrusion detection remains a challenging research area as it involves learning from large-scale imbalanced multiclass datasets. While machine learning algorithms have been widely used for network intrusion detection, most standard techniques cannot achieve consistent good performance across multiple classes. In this paper we proposed a novel ensemble system based on the modified adaptive boosting with area under the curve (M-AdaBoost-A) algorithm to detect network intrusions more effectively. We combined multiple M-AdaBoost-A-based classifiers into an ensemble by employing various strategies, including particle swarm optimization. To the best of our knowledge, this study is the first to utilize the M-AdaBoost-A algorithm, which incorporates the area under the curve into the boosting process for addressing class imbalance in network intrusion detection. Compared with existing standard techniques, our proposed ensemble system achieved superior performance across multiple classes in both 802.11 wireless intrusion detection and traditional enterprise intrusion detection.

Exploiting domain knowledge to address multi-class imbalance and a heterogeneous feature space in classification tasks for manufacturing data

Classification techniques are increasingly adopted for quality control in manufacturing, e.g., to help domain experts identify the cause of quality issues of defective products. However, real-world data often imply a set of analytical challenges, which lead to a reduced classification performance. Major challenges are a high degree of multi-class imbalance within data and a heterogeneous feature space that arises from the variety of underlying products. This paper considers such a challenging use case in the area of End-of-Line testing, i.e., the final functional test of complex products. Existing solutions to classification or data pre-processing only address individual analytical challenges in isolation. We propose a novel classification system that explicitly addresses both challenges of multi-class imbalance and a heterogeneous feature space together. As main contribution, this system exploits domain knowledge to systematically prepare the training data. Based on an experimental evaluation on real-world data, we show that our classification system outperforms any other classification technique in terms of accuracy. Furthermore, we can reduce the amount of rework required to solve a quality issue of a product.

Comparison between Statistical Models and Machine Learning Methods on Classification for Highly Imbalanced Multiclass Kidney Data.

This study aims to compare the classification performance of statistical models on highly imbalanced kidney data. The health examination cohort database provided by the National Health Insurance Service in Korea is utilized to build models with various machine learning methods. The glomerular filtration rate (GFR) is used to diagnose chronic kidney disease (CKD). It is calculated using the Modification of Diet in Renal Disease method and classified into five stages (1, 2, 3A and 3B, 4, and 5). Different CKD stages based on the estimated GFR are considered as six classes of the response variable. This study utilizes two representative generalized linear models for classification, namely, multinomial logistic regression (multinomial LR) and ordinal logistic regression (ordinal LR), as well as two machine learning models, namely, random forest (RF) and autoencoder (AE). The classification performance of the four models is compared in terms of accuracy, sensitivity, specificity, precision, and F1-Measure. To find the best model that classifies CKD stages correctly, the data are divided into a 10-fold dataset with the same rate for each CKD stage. Results indicate that RF and AE show better performance in accuracy than the multinomial and ordinal LR models when classifying the response variable. However, when a highly imbalanced dataset is modeled, the accuracy of the model performance can distort the actual performance. This occurs because accuracy is high even if a statistical model classifies a minority class into a majority class. To solve this problem in performance interpretation, we not only consider accuracy from the confusion matrix but also sensitivity, specificity, precision, and F-1 measure for each class. To present classification performance with a single value for each model, we calculate the macro-average and micro-weighted values for each model. We conclude that AE is the best model classifying CKD stages correctly for all performance indices.

Improved students’ performance prediction for multi-class imbalanced problems using hybrid and ensemble approach in educational data mining

Among the problems raised in the data mining area, the class imbalance is a well-known issue that always occurs. Many researchers studied this issue in several fields using three commonly used techniques: sampling, ensemble, or cost-sensitive learning. However, such studies are still new in education domains. This problem always related to the quality of data that gives the most impact to form an accurate prediction result. Many previous studies focus on binary imbalance classification problems instead of the multi-class imbalance problem in education data. This study used 4413 student instances of two datasets; students’ information system and e-learning from the Faculty of Engineering in a Malaysia university for First Semester 2017/2018. Three sampling categories utilized in this study are oversampling techniques, undersampling techniques, and hybrid techniques. The research empirically analyzes five types of ensemble classifiers and seven sampling techniques. The experimental results show a hybrid technique ROS with AdaBoost produces the most excellent performance compared to the other benchmark techniques. SMOTEENN technique with ensembles classifiers consistently produces high results. This technique has great potential in improving the students’ performance prediction model.