Impact Of Class Imbalance Research Articles

Transfer learning–based PET/CT three-dimensional convolutional neural network fusion of image and clinical information for prediction of EGFR mutation in lung adenocarcinoma

BackgroundTo introduce a three-dimensional convolutional neural network (3D CNN) leveraging transfer learning for fusing PET/CT images and clinical data to predict EGFR mutation status in lung adenocarcinoma (LADC).MethodsRetrospective data from 516 LADC patients, encompassing preoperative PET/CT images, clinical information, and EGFR mutation status, were divided into training (n = 404) and test sets (n = 112). Several deep learning models were developed utilizing transfer learning, involving CT-only and PET-only models. A dual-stream model fusing PET and CT and a three-stream transfer learning model (TS_TL) integrating clinical data were also developed. Image preprocessing includes semi-automatic segmentation, resampling, and image cropping. Considering the impact of class imbalance, the performance of the model was evaluated using ROC curves and AUC values.ResultsTS_TL model demonstrated promising performance in predicting the EGFR mutation status, with an AUC of 0.883 (95%CI = 0.849–0.917) in the training set and 0.730 (95%CI = 0.629–0.830) in the independent test set. Particularly in advanced LADC, the model achieved an AUC of 0.871 (95%CI = 0.823–0.919) in the training set and 0.760 (95%CI = 0.638–0.881) in the test set. The model identified distinct activation areas in solid or subsolid lesions associated with wild and mutant types. Additionally, the patterns captured by the model were significantly altered by effective tyrosine kinase inhibitors treatment, leading to notable changes in predicted mutation probabilities.ConclusionPET/CT deep learning model can act as a tool for predicting EGFR mutation in LADC. Additionally, it offers clinicians insights for treatment decisions through evaluations both before and after treatment.

Impact of class imbalance in VeReMi dataset for misbehavior detection in autonomous vehicles

Impact of class imbalance in VeReMi dataset for misbehavior detection in autonomous vehicles

A novel Random Forest integrated model for imbalanced data classification problem

In recent years, most researchers focused on the classification problems of imbalanced data sets, and these problems are widely distributed in industrial production and medical research fields. For these highly imbalanced data sets, the ensemble method based on over-sampling is one of the most competitive techniques in the present research. However, the incorrect sampling strategy easily affected the model performance, which increased the training complexity and caused an over-fitting problem. This article proposed an equilibrium ensemble method (DCI-ISSA) with two novel techniques to conquer these shortcomings. Firstly, this paper raised an over-sampling approach (Data Center Interpolation DCI) to offer a counterbalanced data set for the single learner, which can prevent the base learners from the impact of class imbalance. Additionally, we provided a parameter optimization method for Random Forest (RF), which used the Improved Sparrow Search Algorithm (ISSA) to find the optimal parameters for different imbalanced data sets dynamically. These parameters can improve the classification performance of base classifiers and adjust to all kinds of lopsided data sets with distinct sizes. Experimental results showed that the DCI-ISSA-RF model outperforms other famous approaches for the imbalanced data sets with various dimensions.

Perception model of surrounding rock geological conditions based on TBM operational big data and combined unsupervised-supervised learning

The perception of surrounding rock geological conditions ahead the tunnel face is essential for TBM safe and efficient tunnelling. This paper developed a perception approach of surrounding rock class based on TBM operational big data and combined unsupervised-supervised learning. In data preprocessing, four data mining techniques (i.e., Z-score, K-NN, Kalman filtering, and wavelet packet decomposition) were used to detect outliers, substitute outliers, suppress noise, and extract features, respectively. Then, GMM was used to revise the original surrounding rock class through clustering TBM load parameters and performance parameters in view of the shortcomings of the HC method in the TBM-excavated tunnel. After that, five various ensemble learning classification models were constructed to identify the surrounding rock class, in which model hyper-parameters were automatically tuned by Bayes optimization. In order to evaluate model performance, balanced accuracy, Kappa, F1-score, and training time were taken into account, and a novel multi-metric comprehensive ranking system was designed. Engineering application results indicated that LightGBM achieved the most superior performance with the highest comprehensive score of 6.9066, followed by GBDT (5.9228), XGBoost (5.4964), RF (3.7581), and AdaBoost (0.9946). Through the weighted purity reduction algorithm, the contributions of input features on the five models were quantitatively analyzed. Finally, the impact of class imbalance on model performance was discussed using the ADASYN algorithm, showing that eliminating class imbalance can further improve the model's perception ability.

Probability Density Machine: A New Solution of Class Imbalance Learning

Class imbalance learning (CIL) is an important branch of machine learning as, in general, it is difficult for classification models to learn from imbalanced data; meanwhile, skewed data distribution frequently exists in various real-world applications. In this paper, we introduce a novel solution of CIL called Probability Density Machine (PDM). First, in the context of Gaussian Naive Bayes (GNB) predictive model, we analyze the reason why imbalanced data distribution makes the performance of predictive model decline in theory and draw a conclusion regarding the impact of class imbalance that is only associated with the prior probability, but does not relate to the conditional probability of training data. Then, in such context, we show the rationality of several traditional CIL techniques. Furthermore, we indicate the drawback of combining GNB with these traditional CIL techniques. Next, profiting from the idea of K-nearest neighbors probability density estimation (KNN-PDE), we propose the PDM which is an improved GNB-based CIL algorithm. Finally, we conduct experiments on lots of class imbalance data sets, and the proposed PDM algorithm shows the promising results.

Strength of Stacking Technique of Ensemble Learning in Rockburst Prediction with Imbalanced Data: Comparison of Eight Single and Ensemble Models

Rockburst is a common dynamic geological hazard, severely restricting the development and utilization of underground space and resources. As the depth of excavation and mining increases, rockburst tends to occur frequently. Hence, it is necessary to carry out a study on rockburst prediction. Due to the nonlinear relationship between rockburst and its influencing factors, artificial intelligence was introduced. However, the collected data were typically imbalanced. Single algorithms trained by such data have low recognition for minority classes. In order to handle the problem, this paper employed stacking technique of ensemble learning to establish rockburst prediction models. In total, 246 sets of data were collected. In the preprocessing stage, three data mining techniques including principal component analysis, local outlier factor and expectation maximization algorithm were used for dimension reduction, outlier detection and outlier substitution, respectively. Then, the pre-processed data were split into a training set (75%) and a test set (25%) with stratified sampling. Based on the four classical single intelligent algorithms, namely k-nearest neighbors (KNN), support vector machine (SVM), deep neural network (DNN) and recurrent neural network (RNN), four ensemble models (KNN–RNN, SVM–RNN, DNN–RNN and KNN–SVM–DNN–RNN) were built by stacking technique of ensemble learning. The prediction performance of eight models was evaluated, and the differences between single models and ensemble models were analyzed. Additionally, a sensitivity analysis was conducted, revealing the importance of input variables on the models. Finally, the impact of class imbalance on the prediction accuracy and fitting effect of models was quantitatively discussed. The results showed that stacking technique of ensemble learning provides a new and promising way for rockburst prediction, which exhibits unique advantages especially when using imbalanced data.

Dual cost‐sensitivity factors‐based power system transient stability assessment

In actual power system operation, there is a significant imbalance between the number of stable and unstable samples, and the misclassification costs of the two classes of samples are different. In addition to the class imbalance, there is another imbalance of regional samples in the feature space. In order to reduce the impact of class imbalance and regional imbalance on the performance of the model, a method of dual cost-sensitivity factors for transient stability assessment is proposed. The method sets the balance factor and modulating factor in the loss function of light gradient boosting machine. The former corrects the classification bias caused by class imbalance, and the latter focuses on improving the classification accuracy of overlapping area samples. Combining the two factors, the model not only improves the accuracy of unstable samples, but also reduces the misjudgement of stable samples. In online application, a fast update scheme with memory function is proposed. In this scheme, incremental learning is used to update the model with fewer samples and less computational time, so as to achieve better evaluation performance. Case studies on three power systems demonstrate the generalisation performance of the proposed model and the effectiveness of the update scheme.

Rethinking labelling in road segmentation

ABSTRACTDeep models are extremely data hungry. Their success is being driven by the availability of large amounts of data for training. For semantic segmentation tasks on aerial and satellite imagery, a major dilemma at present is that it still relies heavily on manual labelling of data. Among these tasks, the semantic segmentation of road is special since it is possible to use auxiliary data, such as GPS track data, to automatically label data. For a better understanding of this possibility, this paper proposes to rethink some basic issues of labelling approaches for roads.We experimentally investigated the unavoidable class imbalance problem in road segmentation tasks through simulated and real datasets and quantitatively show that class imbalance has a serious detrimental impact on deep model’s generalization performance. We also observed that the detrimental impact even outweighs the benefits of strictly annotating roads – expanding road labels can give deep networks better segmentation accuracy, even though the segmentation location is no longer the edge of the road. We think this is because the impact of class imbalance is much overwhelming than the sensitivity of DNN on the edges of real roads. This finding is valuable for supporting the use of centreline-based approaches in place of edge-based approaches in some applications for better cost-effective solutions.We proposed a guided Random Sample Consensus (RANSAC) algorithm to determine the optimal expansion ratio of road label. On these bases, we further proposed a general framework to combine two networks to achieve better performance than the state-of-the-art performance of using alone. We attribute this to the alleviation of the class imbalance problem because simply cascading the two networks does not achieve the purpose of improving accuracy in our experiments. We believe that this work is enlightening for studies of road segmentation.

The impact of class imbalance in classification performance metrics based on the binary confusion matrix

A major issue in the classification of class imbalanced datasets involves the determination of the most suitable performance metrics to be used. In previous work using several examples, it has been shown that imbalance can exert a major impact on the value and meaning of accuracy and on certain other well-known performance metrics. In this paper, our approach goes beyond simply studying case studies and develops a systematic analysis of this impact by simulating the results obtained using binary classifiers. A set of functions and numerical indicators are attained which enables the comparison of the behaviour of several performance metrics based on the binary confusion matrix when they are faced with imbalanced datasets. Throughout the paper, a new way to measure the imbalance is defined which surpasses the Imbalance Ratio used in previous studies. From the simulation results, several clusters of performance metrics have been identified that involve the use of Geometric Mean or Bookmaker Informedness as the best null-biased metrics if their focus on classification successes (dismissing the errors) presents no limitation for the specific application where they are used. However, if classification errors must also be considered, then the Matthews Correlation Coefficient arises as the best choice. Finally, a set of null-biased multi-perspective Class Balance Metrics is proposed which extends the concept of Class Balance Accuracy to other performance metrics.

Impact of class imbalance on personalized program guide performance

When TV recommender systems perform well, number of interactions in which their users expressed positive feedback on the recommended content is expected to be greater than the number of negative ones. This is known as class imbalance and, paradoxically, it degrades the system performance by making the identification of the programs the user will dislike increasingly difficult. As the misclassification of the unwanted content is easily perceived by TV viewers, it should be avoided by all means. In this paper, a personalized TV program guide based on neural network is described. It is shown how class imbalance information can be exploited in learning the user preferences. This not only improves the system performance, but increases the user satisfaction as well.1

Impact of class-imbalance on multi-class high-dimensional class prediction

The goal of multi-class supervised classification is to develop a rule that accurately predicts the class membership of new samples when the number of classes is larger than two. In this paper we consider high-dimensional class-imbalanced data: the number of variables greatly exceeds the number of samples and the number of samples in each class is not equal. We focus on Friedman's one-versus-one approach for three-class problems and show how its class probabilities depend on the class probabilities from the binary classification sub-problems. We further explore its performance using diagonal linear discriminant analysis (DLDA) as a base classifier and compare its performance with multi-class DLDA, using simulated and real data. Our results show that the class-imbalance has a significant effect on the classification results: the classification is biased towards the majority class as in the two-class problems and the problem is magnified when the number of variables is large. The amount of the bias depends also, jointly, on the magnitude of the differences between the classes and on the sample size: the bias diminishes when the difference between the classes is larger or the sample size is increased. Also variable selection plays an important role in the class-imbalance problem and the most effective strategy depends on the type of differences that exist between classes. DLDA seems to be among the least sensible classifiers to class-imbalance and its use is recommended also for multi-class problems. Whenever possible the experiments should be planned using balanced data in order to avoid the class-imbalance problem.