Accuracy Of Minority Class Research Articles

Class imbalance: A crucial factor affecting the performance of tea plantations mapping by machine learning

Due to disparities in area among various land cover types, class imbalance has always existed in crop mapping research, posing uncertainties in extracting minority classes which occupy a smaller area. In this paper, taking tea plantations mapping in Hangzhou city as an example, we created a series of training datasets with different imbalance-ratios (IRs), compared the accuracy between the extraction models using these datasets, and analyzed the impact of class imbalance on various machine learning algorithms (Artificial Neural Network, Decision Tree, Random Forest and XGBoost), aiming to provide a feasible approach to improve the mapping accuracy of minority classes. The leave-one-out cross validation results showed that in most cases, with the increase of the IR, the model’s F2-score first increased and then decreased, and the increase of F2-scores ranged from 0.2% to 29.2%, suggesting that moderately increasing the number of other samples in the training dataset can improve the tea plantations extraction accuracy. Consistent result can also be obtained by using the whole city’s samples for modeling and random sampling validation. XGBoost performed best among the four algorithms, which yielded the optimal tea plantations map with a PA of 97%, UA of 93% and F2-score of 96% when the IR of the training dataset was 6. The UA was 19% higher than that of the model using a balanced dataset (IR=1) and was 11% higher than that of the model using pseudo-balanced datasets created by the oversampling method. The conclusions of this study offer insights for the identification of minority classes, contributing to achieving higher accuracy in remote sensing crop mapping.

A Comparative Analysis of Combination of CNN-Based Models with Ensemble Learning on Imbalanced Data

This study investigates the usefulness of the Synthetic Minority Oversampling Technique (SMOTE) in conjunction with convolutional neural network (CNN) models, which include both single and ensemble classifiers. The objective of this research is to handle the difficulty of multi-class imbalanced image classification. The application of SMOTE in imbalanced picture datasets is still underexplored, even though CNNs have been shown to be successful in image classification and that ensemble learning approaches have improved their performance. To investigate whether or not SMOTE can increase classification accuracy and other performance measures when combined with CNN-based classifiers, our research makes use of a CIFAR-10 dataset that has been artificially step-imbalanced and has varying imbalanced ratios. We conducted experiments using five distinct models, namely AdaBoost, XGBoost, standalone CNN, CNN-AdaBoost, and CNN-XGBoost, on datasets that were either imbalanced or SMOTE-balanced. Metrics such as accuracy, precision, recall, F1-score, and the area under the receiver operating characteristic curve (AUC) were included in the evaluation process. The findings indicate that SMOTE dramatically improves the accuracy of minority classes, and that the combination of ensemble classifiers with CNNs and oversampling techniques significantly improves overall classification performance, particularly in situations when there is a high-class imbalance. When it comes to enhancing imbalanced classification tasks, this study demonstrates the potential of merging oversampling techniques with CNN-based ensemble classifiers to minimize the impacts of class imbalance in picture datasets. This suggests a promising direction for future research in this area.

RL-ECGNet: resource-aware multi-class detection of arrhythmia through reinforcement learning

Arrhythmia is a fatal cardiac clinical condition that risks the lives of millions every year. It has multiple classes with variable prevalence rates. Some rare arrhythmia classes are equally critical as common ones, yet are very hard to detect due to limited training samples. While several methods accurately detect Arrhythmia's multi-class, minority class accuracy remains low and these methods are resource-intensive. Therefore, most of the existing detection systems ignore minority classes in their classification or focus on binary classification. In this study, we introduce RL-ECGNet, a resource-efficient reinforcement learning-based optimization for multi-class arrhythmia detection, encompassing minority classes, through ECG signal analysis. RL-ECGNet uses raw ECG signals, processes them to extract the temporal ECG features, and utilizes Reinforcement Learning (RL) to optimize the training and network hyperparameters of the Deep Learning (DL) models while reducing resource consumption. For evaluation, four DL models, namely, MLP, CNN, LSTM, and GRU, are trained and optimized. Moreover, time and memory usage are minimized to optimize resource consumption. Throughout the evaluation of the four DL models, the proposed RL model achieved accuracies ranging from 88.45% to 96.41% for all 9 arrhythmia classes, including minority classes. In addition, the proposed RL method improved performance by a factor ranging from 1.28 to 1.39 in terms of accuracy. Moreover, the optimized DL models had reduced training time, as well as minimized memory usage. The proposed method achieved resource consumption reduction ranging from 1.36 to 1.925 times for training time, and from 1.179 to 1.815 times for memory usage.

An Efficient Data Augmentation Method for Automatic Modulation Recognition from Low-Data Imbalanced-Class Regime

The application of deep neural networks to address automatic modulation recognition (AMR) challenges has gained increasing popularity. Despite the outstanding capability of deep learning in automatic feature extraction, predictions based on low-data regimes with imbalanced classes of modulation signals generally result in low accuracy due to an insufficient number of training examples, which hinders the wide adoption of deep learning methods in practical applications of AMR. The identification of the minority class of samples can be crucial, as they tend to be of higher value. However, in AMR tasks, there is a lack of attention and effective solutions to the problem of Imbalanced-class in a low-data regime. In this work, we present a practical automatic data augmentation method for radio signals, called SigAugment, which incorporates eight individual transformations and effectively improves the performance of AMR tasks without additional searches. It surpasses existing data augmentation methods and mainstream methods for solving low-data and imbalanced-class problems on multiple datasets. By simply embedding SigAugment into the training pipeline of an existing model, it can achieve state-of-the-art performance on benchmark datasets and dramatically improve the classification accuracy of minority classes in the low-data imbalanced-class regime. SigAugment can be trained for uniform use on different types of models and datasets and works right out of the box.

Real-time grouping of tobacco through channel weighting and dynamic loss regulation

Grouping of tobacco is a key technology in intelligent management of tobacco planting and fine blending in cigarette industry. However, existing grouping methods based on deep learning are limited by lacking key feature expression due to excessive abstraction at higher scales. In addition, the inter-class discrimination ability is limited and the model prefers to majority classes during training. To tackle these problems, we propose a real-time tobacco grouping network (TGNet) based on channel weighting of higher scale features and dynamic loss regulation. Firstly, the industrial array camera equipped on a machine vision detection platform is used to capture the original tobacco images, which including the orange (F), lemon (L), variegated (K), greenish (V), and green-yellow (GY) categories manually labeled by grading experts. Then, to tackle the deficiency of key feature expression in the higher scale abstraction layers, a feature channel weighting module is proposed, which generates weighting factors proportional to different channels to re-encode higher scale features and emphatically highlight the key area and color information. The lightweight structure of feature channel weighting module compresses the model volume and improves forward inference speed. Finally, aiming at the problems of inter-group ambiguity and class imbalance dilemma, the focal and dynamic margin (FADM) loss with focusing on minority classes training and adaptative margin is proposed. The dynamic margin is adaptatively encoded by a Cosine function of similarity, which uses tobacco category recognition probability to measure the similarity of tobacco. Extensive performance evaluations show that TGNet outperforms other classification models in terms of the average classification accuracy 77.16%, inference time 112FPS, model size 2.5MB, model computation 0.12GFlops and model parameter number 0.613 M. The classification accuracy of minority classes V and GY is improved.

Comparison of common classification strategies for large-scale vegetation mapping over the Google Earth Engine platform

• We assessed the effect of various sample properties on classification performance. • The use of pixel time series from fusion data achieved optimal results. • Target class proportions and sample time frames affected accuracy. • Suboptimal methods had a larger impact on the accuracy of minority classes. • The combined use of Sentinel MSI and SAR data provided accurate vegetation maps. Vegetation resources have an essential role in sustainable development due to their close relationship with natural resource management and environmental protection. The monitoring of land use and cover is key for a more sustainable management of these resources, and Earth Observation satellites have provided an increasingly powerful platform for performing this task. To date, numerous classification algorithms have been developed for vegetation mapping, but a comparative evaluation of the strategies commonly used in large-scale applications (50000 km 2 and above) is lacking. We developed a classification framework based on random forests and Sentinel data within the Google Earth Engine platform to assess the performance of various strategies over a complex landscape with a wide range of vegetation classes, plot configurations, and agricultural practices. These strategies differed in key aspects related to the characteristics of the classifier and the sample, and were evaluated by class-specific and general accuracy metrics. We found that the use of pixel time series features from fusion data, class-balanced labels, and multi-season time frames enhanced overall performance by 1.3%–14.1% over alternative approaches, but in some cases also generated tradeoffs of 1.6%–8.5% between recall and precision. In general, suboptimal strategies were particularly ineffective for the detection of infrequent classes. Finally, the different parameter values used in the random forests did not have a significant influence over the results. Our results demonstrate the importance of algorithm design in the effective classification of multiple vegetation classes, corroborate the usefulness of Sentinel data to generate mapping products of high resolution and accuracy, and highlight the importance of cloud computing tools for the development of vegetation mapping tools in large-scale applications. These findings provide general guidelines for the design of future classification frameworks, which are necessary for facilitating a sustainable management of natural resources worldwide.

The prediction of cardiac abnormality and enhancement in minority class accuracy from imbalanced ECG signals using modified deep neural network models

Cardiovascular disease (CVD) is the most fatal disease in the world, so its accurate and automated detection in the early stages will certainly support the medical expert in timely diagnosis and treatment, which can save many lives. Many types of research have been carried out in this regard, but due to the problem of data imbalance in the medical and health care sector, it may not provide the desired results in all aspects. To overcome this problem, a sequential ensemble technique has been proposed that detects 6 types of cardiac arrhythmias on large ECG imbalanced datasets, and the data imbalanced issue of the ECG dataset has been addressed by using a hybrid data resampling technique called “Synthetically Minority Oversampling Technique and Tomek Link (SMOTE + Tomek)". The sequential ensemble technique employs two distinct deep learning models: Convolutional Neural Network (CNN) and a hybrid model, CNN with Long Short-Term Memory Network (CNN-LSTM). The two standard datasets “MIT-BIH arrhythmias database” (MITDB) and “PTB diagnostic database” (PTBDB) are combined and extracted 23, 998 ECG beats for the model validation. In this work, the three models CNN, CNN-LSTM, and ensemble approach were tested on four kinds of ECG datasets: the original data (imbalanced), the data sampled using a random oversampled technique, data sampled using SMOTE, and the dataset resampled using SMOTE + Tomek algorithm. The overall highest accuracy was obtained of 99.02% on the SMOTE + Tomek sampled dataset by ensemble technique and the minority class accuracy result (Recall) is improved by 20% as compared to the imbalanced data.

An efficiency curve for evaluating imbalanced classifiers considering intrinsic data characteristics: Experimental analysis

Balancing the accuracy rates of the majority and minority classes is challenging in imbalanced classification. Furthermore, data characteristics have a significant impact on the performance of imbalanced classifiers, which are generally neglected by existing evaluation methods. The objective of this study is to introduce a new criterion to comprehensively evaluate imbalanced classifiers. Specifically, we introduce an efficiency curve that is established using data envelopment analysis without explicit inputs (DEA-WEI), to determine the trade-off between the benefits of improved minority class accuracy and the cost of reduced majority class accuracy. In sequence, we analyze the impact of the imbalanced ratio and typical imbalanced data characteristics on the efficiency of the classifiers. Empirical analyses using 68 imbalanced data reveal that traditional classifiers such as C4.5 and the k-nearest neighbor are more effective on disjunct data, whereas ensemble and undersampling techniques are more effective for overlapping and noisy data. The efficiency of cost-sensitive classifiers decreases dramatically when the imbalanced ratio increases. Finally, we investigate the reasons for the different efficiencies of classifiers on imbalanced data and recommend steps to select appropriate classifiers for imbalanced data based on data characteristics.

Deep transferable learning on heartbeat classification for imbalance dataset

Electrocardiogram (ECG) data recorded by medical devices are hard to analyze manually. Therefore, it is important to analyze and categorize each heartbeat using machine learning. Recently, advancements in machine learning have made classification of complex data easy and fast. However, these machine learning algorithms require sufficient amount of training data and have limited performance in case the data is imbalance. In case of MIT-BIH arrhythmia dataset, the distribution of training instances are quite imbalance. Many machine learning, particularly deep learning, algorithms give high accuracy on these datasets but still the minority classes have zero accuracy. In this paper, we improve the accuracy of minority classes without hurting the overall accuracy of other classes using transfer learning. The accuracy of existing deep learning model is increased from 90.67% to 98.47%, respectively.

Overcoming the ordinal imbalanced data problem by combining data processing and stacked generalizations

Ordinal imbalanced datasets are pervasive in real world applications but remain challenging to analyse as they require specific methods to account for the ordering information and imbalanced classes. Failure to account for both those characteristics can substantially impact the model predictive performance. However, existing methods tend to focus either on ordinality or imbalance, rather than addressing both simultaneously. The few approaches that do account for both characteristics are not always easy to implement for non-advanced analysts and simpler approaches are needed to facilitate appropriate data processing. Here, we developed a general approach using some of the most popular machine learning algorithms to ensure appropriate processing of ordinal imbalanced datasets and to optimize the predictions of all classes. After transforming the multi-class ordinal problem into a well-known binary problem, we implemented several different resampling methods in a decision-tree classifier. We then used a stacked generalization algorithm to combine the classifiers to improve model predictive performance. To test our approach, we used two ordinal imbalanced datasets on student performance and wine quality. Individual resampling techniques tended to improve the accuracy of minority classes, while simultaneously increasing the number of false positives in those classes. This resulted in a decrease, sometimes substantial, in accuracy of other classes. The stacking model offered a good compromise between improvement in accuracy of minority classes and mitigation of reduced accuracy in other classes. Our approach provided useful insights into modelling strategies that should be favoured for implementation in production that involve these common datasets, depending on the end-user interests.

Tversky Similarity based UnderSampling with Gaussian Kernelized Decision Stump Adaboost Algorithm for Imbalanced Medical Data Classification

In recent years, imbalanced data classification are utilized in several domains including, detecting fraudulent activities in banking sector, disease prediction in healthcare sector and so on. To solve the Imbalanced classification problem at data level, strategy such as undersampling or oversampling are widely used. Sampling technique pose a challenge of significant information loss. The proposed method involves two processes namely, undersampling and classification. First, undersampling is performed by means of Tversky Similarity Indexive Regression model. Here, regression along with the Tversky similarity index is used in analyzing the relationship between two instances from the dataset. Next, Gaussian Kernelized Decision stump AdaBoosting is used for classifying the instances into two classes. Here, the root node in the Decision Stump takes a decision on the basis of the Gaussian Kernel function, considering average of neighboring points accordingly the results is obtained at the leaf node. Weights are also adjusted to minimizing the training errors occurring during classification to find the best classifier. Experimental assessment is performed with two different imbalanced dataset (Pima Indian diabetes and Hepatitis dataset). Various performance metrics such as precision, recall, AUC under ROC score and F1-score are compared with the existing undersampling methods. Experimental results showed that prediction accuracy of minority class has improved and therefore minimizing false positive and false negative.

Molecular Classification and Interpretation of Amyotrophic Lateral Sclerosis Using Deep Convolution Neural Networks and Shapley Values.

Amyotrophic lateral sclerosis (ALS) is a prototypical neurodegenerative disease characterized by progressive degeneration of motor neurons to severely effect the functionality to control voluntary muscle movement. Most of the non-additive genetic aberrations responsible for ALS make its molecular classification very challenging along with limited sample size, curse of dimensionality, class imbalance and noise in the data. Deep learning methods have been successful in many other related areas but have low minority class accuracy and suffer from the lack of explainability when used directly with RNA expression features for ALS molecular classification. In this paper, we propose a deep-learning-based molecular ALS classification and interpretation framework. Our framework is based on training a convolution neural network (CNN) on images obtained from converting RNA expression values into pixels based on DeepInsight similarity technique. Then, we employed Shapley additive explanations (SHAP) to extract pixels with higher relevance to ALS classifications. These pixels were mapped back to the genes which made them up. This enabled us to classify ALS samples with high accuracy for a minority class along with identifying genes that might be playing an important role in ALS molecular classifications. Taken together with RNA expression images classified with CNN, our preliminary analysis of the genes identified by SHAP interpretation demonstrate the value of utilizing Machine Learning to perform molecular classification of ALS and uncover disease-associated genes.

Cost-Sensitive Extremely Randomized Trees Algorithm for Online Fault Detection of Wind Turbine Generators

The number of normal samples of wind turbine generators is much larger than the number of fault samples. To solve the problem of imbalanced classification in wind turbine generator fault detection, a cost-sensitive extremely randomized trees (CS-ERT) algorithm is proposed in this paper, in which the cost-sensitive learning method is introduced into an extremely randomized trees (ERT) algorithm. Based on the classification misclassification cost and class distribution, the misclassification cost gain (MCG) is proposed as the score measure of the CS-ERT model growth process to improve the classification accuracy of minority classes. The Hilbert-Schmidt independence criterion lasso (HSICLasso) feature selection method is used to select strongly correlated non-redundant features of doubly-fed wind turbine generators. The effectiveness of the method was verified by experiments on four different failure datasets of wind turbine generators. The experiment results show that average missing detection rate, average misclassification cost and gMean of the improved algorithm better than those of the ERT algorithm. In addition, compared with the CSForest, AdaCost and MetaCost methods, the proposed method has better real-time fault detection performance.

Classification of Imbalanced Data Using Deep Learning with Adding Noise

This paper proposes a method to treat the classification of imbalanced data by adding noise to the feature space of convolutional neural network (CNN) without changing a data set (ratio of majority and minority data). Besides, a hybrid loss function of crossentropy and KL divergence is proposed. The proposed approach can improve the accuracy of minority class in the testing data. In addition, a simple design method for selecting structure of CNN is first introduced and then, we add noise in feature space of CNN to obtain proper features by a training process and to improve the classification results. From comparison results, we can find that the proposed method can extract the suitable features to improve the accuracy of minority class. Finally, illustrated examples of multiclass classification problems and the corresponding discussion in balance ratio are presented. Our approach performs well with smaller network structure compared with other deep models. In addition, the performance is improved over 40% in defective accuracy by adding noise approach. Finally, the accuracy is higher than 96%; even the imbalanced ratio (IR) is one hundred.

Enhanced-Random-Feature-Subspace-Based Ensemble CNN for the Imbalanced Hyperspectral Image Classification

Hyperspectral image (HSI) classification often faces the problem of multiclass imbalance, which is considered to be one of the major challenges in the field of remote sensing. In recent years, deep learning has been successfully applied to the HSI classification, a convolutional neural network (CNN) is one of the most representative of them. However, it is difficult to effectively improve the accuracy of minority classes under the problem of multiclass imbalance. In addition, ensemble learning has been successfully applied to solve multiclass imbalance, such as random forest (RF) This article proposes a novel enhanced-random-feature-subspace-based ensemble CNN algorithm for the multiclass imbalanced problem. The main idea is to perform random oversampling of training samples and multiple data enhancements based on random feature subspace, and then, construct an ensemble learning model combining random feature selection and CNN to the HSI classification. Experimental results on three public hyperspectral datasets show that the performance of the proposed method is better than the traditional CNN, RF, and deep learning ensemble methods.

Generative Adversarial Network for Class-Conditional Data Augmentation

We propose a novel generative adversarial network for class-conditional data augmentation (i.e., GANDA) to mitigate data imbalance problems in image classification tasks. The proposed GANDA generates minority class data by exploiting majority class information to enhance the classification accuracy of minority classes. For stable GAN training, we introduce a new denoising autoencoder initialization with explicit class conditioning in the latent space, which enables the generation of definite samples. The generated samples are visually realistic and have a high resolution. Experimental results demonstrate that the proposed GANDA can considerably improve classification accuracy, especially when datasets are highly imbalanced on standard benchmark datasets (i.e., MNIST and CelebA). Our generated samples can be easily used to train conventional classifiers to enhance their classification accuracy.

Recognition of imbalanced underwater acoustic datasets with exponentially weighted cross-entropy loss

Class imbalance, an objective problem of underwater acoustic datasets, has hardly been paid attention to, but often results in low recognition accuracy of minority classes. The main objective of this paper is to provide an effective method for the recognition of imbalanced underwater acoustic datasets. For this purpose, an exponentially weighted cross-entropy loss is proposed as the convolutional neural network’s loss function, which adds an impact factor to the standard cross-entropy loss according to the prediction probability of each sample. The proposed approach is evaluated on imbalanced underwater acoustic datasets of targets and communication signals. Recognition accuracy has been increased by 6.67% and 13.33% for underwater targets, 25% and 2.5% for underwater simulation communication signals, and 12.5% for underwater experimental communication signals, compared with the cross-entropy loss and the focal loss. Results on simulation data and experimental data show that the proposed approach can obtain higher recognition accuracy than the cross-entropy loss and the focal loss, which provides evidence for its effectiveness.

Predicting Clinical Outcome in Acute Ischemic Stroke Using Parallel Multi-parametric Feature Embedded Siamese Network.

Stroke is the second leading cause of death and disability worldwide, with ischemic stroke as the most common type. The preferred diagnostic procedure at the acute stage is the acquisition of multi-parametric magnetic resonance imaging (MRI). This type of imaging not only detects and locates the stroke lesion, but also provides the blood flow dynamics that helps clinicians in assessing the risks and benefits of reperfusion therapies. However, evaluating the outcome of these risky therapies beforehand is a complicated task due to the variability of lesion location, size, shape, and cerebral hemodynamics involved. Though the fully automated model for predicting treatment outcomes using multi-parametric imaging would be highly valuable in clinical settings, MRI datasets acquired at the acute stage are mostly scarce and suffer high class imbalance. In this paper, parallel multi-parametric feature embedded siamese network (PMFE-SN) is proposed that can learn with few samples and can handle skewness in multi-parametric MRI data. Moreover, five suitable evaluation metrics that are insensitive to imbalance are defined for this problem. The results show that PMFE-SN not only outperforms other state-of-the-art techniques in all these metrics but also can predict the class with a small number of samples, as well as the class with high number of samples. An accuracy of 0.67 on leave one cross out testing has been achieved with only two samples (minority class) for training and accuracy of 0.61 with the highest number of samples (majority class). In comparison, state-of-the-art using hand crafted features has 0 accuracy for minority class and 0.33 accuracy for majority class.

A Hybrid Data Balancing Method for Classification of Imbalanced Training Data within Google Earth Engine: Case Studies from Mountainous Regions

Distribution of Land Cover (LC) classes is mostly imbalanced with some majority LC classes dominating against minority classes in mountainous areas. Although standard Machine Learning (ML) classifiers can achieve high accuracies for majority classes, they largely fail to provide reasonable accuracies for minority classes. This is mainly due to the class imbalance problem. In this study, a hybrid data balancing method, called the Partial Random Over-Sampling and Random Under-Sampling (PROSRUS), was proposed to resolve the class imbalance issue. Unlike most data balancing techniques which seek to fully balance datasets, PROSRUS uses a partial balancing approach with hundreds of fractions for majority and minority classes to balance datasets. For this, time-series of Landsat-8 and SRTM topographic data along with various spectral indices and topographic data were used over three mountainous sites within the Google Earth Engine (GEE) cloud platform. It was observed that PROSRUS had better performance than several other balancing methods and increased the accuracy of minority classes without a reduction in overall classification accuracy. Furthermore, adopting complementary information, particularly topographic data, considerably increased the accuracy of minority classes in mountainous areas. Finally, the obtained results from PROSRUS indicated that every imbalanced dataset requires a specific fraction(s) for addressing the class imbalance problem, because different datasets contain various characteristics.

Data stream classification: a review

The tremendous amount of data is generated regularly through areas like networking, telecommunication, stock market, satellite, weather forecasting, etc. So, the classification process becomes important to extract knowledge from such a huge amount of data. The handling of concept drifting data stream and skewed data classification are the major issues and challenges in the data streams mining field. In the presence of concept drift, the performance of the learning algorithm always degrades. On the other side in skewed data problems, majority class accuracy always dominates minority class accuracy which generates the wrong result. This paper discusses the implemented methods which worked for skewed data and concept drifting data as well as their merits and demerits. This paper focused on metrics used in the classification process and issues that arise while learning with skewed and concept drifting data.