Class Imbalance Problem Research Articles

Predicting the potential toxicity of the metal oxide nanoparticles using machine learning algorithms

Over the years, machine learning (ML) algorithms have proven their ability to make reliable predictions of the toxicity of metal oxide nanoparticles. This paper proposed a predictive ML model of the potential toxicity of metal oxide nanoparticles. A dataset consisting of 79 descriptors including 24 metal oxide nanoparticles (MexOy NPs) and their physicochemical and structural characteristics is adopted. The proposed model comprises of three main phases. The first phase is used to analyze the characteristics of nanoparticles along with their toxicity behavior. In the second phase, the problems associated with the metal oxide nanoparticles dataset are tackled. The first problem namely the class imbalance problem is handled through utilizing synthetic minority over-sampling technique (SMOTE). The second problem namely the outliers is handled through applying a novel feature selection algorithm based on the enhanced binary version of the sine tree-seed algorithm (EBSTSA). The proposed EBSTSA is used to find the relevant features affecting toxicity. The density-based spatial clustering of applications with noise (DBSCAN) is utilized as a tool for identifying outliers in the dataset and for visualizing the impact of the feature selection on the performance of the subsequent classification. Finally, in the third phase, the support vector machine (SVM) supervised machine learning algorithm and k-fold cross-validation method are applied to classify the mode of action of each instance of nanoparticle as toxic or nontoxic. The simulation results showed that the EBSTSA-based feature selection algorithm is reliable and robust across 23 benchmark datasets from the UCI machine learning repository. The results also showed that proposed EBSTSA can effectively find the relevant descriptors for nano-particles. Furthermore, the results demonstrated the efficacy of the proposed ML toxicity prediction model. It is obtained on average 1.02% of error rate, 100% of specificity, 98.87% of sensitivity, and 99.47% of f1-score.

Binary classification with fuzzy logistic regression under class imbalance and complete separation in clinical studies

BackgroundIn binary classification for clinical studies, an imbalanced distribution of cases to classes and an extreme association level between the binary dependent variable and a subset of independent variables can create significant classification problems. These crucial issues, namely class imbalance and complete separation, lead to classification inaccuracy and biased results in clinical studies.MethodTo deal with class imbalance and complete separation problems, we propose using a fuzzy logistic regression framework for binary classification. Fuzzy logistic regression incorporates combinations of triangular fuzzy numbers for the coefficients, inputs, and outputs and produces crisp classification results. The fuzzy logistic regression framework shows strong classification performance due to fuzzy logic’s better handling of imbalance and separation issues. Hence, classification accuracy is improved, mitigating the risk of misclassified conditions and biased insights for clinical study patients.ResultsThe performance of the fuzzy logistic regression model is assessed on twelve binary classification problems with clinical datasets. The model has consistently high sensitivity, specificity, F1, precision, and Mathew’s correlation coefficient scores across all clinical datasets. There is no evidence of impact from the imbalance or separation that exists in the datasets. Furthermore, we compare the fuzzy logistic regression classification performance against two versions of classical logistic regression and six different benchmark sources in the literature. These six sources provide a total of ten different proposed methodologies, and the comparison occurs by calculating the same set of classification performance scores for each method. Either imbalance or separation impacts seven out of ten methodologies. The remaining three produce better classification performance in their respective clinical studies. However, these are all outperformed by the fuzzy logistic regression framework.ConclusionFuzzy logistic regression showcases strong performance against imbalance and separation, providing accurate predictions and, hence, informative insights for classifying patients in clinical studies.

Weighted Dual-Channel Model for Arrhythmia Detection using Single Electrocardiogram Signal

The single-lead method plays a central role in arrhythmia warning, and its anomaly detection is of practical significance for the prevention of heart diseases. However, the detection performance is often limited due to the limited information content of single-lead electrocardiogram (ECG) signals. To address this issue, this paper proposes a weighted dualchannel arrhythmia classification model aimed at improving the detection performance of single-lead ECG signals. The model employs a parallel structure of CNN-LSTM and Transformer to capture both local and global features of the ECG signals. By applying weighted learning to the features from the two channels, the model achieves precise arrhythmia classification. Additionally, to alleviate the class imbalance problem, a weighted cross-entropy loss function is introduced, further enhancing the classification performance. Experimental results based on the MIT-BIH arrhythmia database demonstrate that the proposed model exhibits excellent performance in terms of overall accuracy, F1 score, and sensitivity, achieving 99.33%, 94.71%, and 99.33%, respectively. Compared with existing classification models, the proposed model shows superior classification performance and significant practical value, providing solid technical support for the auxiliary diagnosis of arrhythmias and bringing new possibilities for actual clinical applications.

Investigating the Effect of Class Balancing Methods on the Performance of Machine Learning Techniques: Credit Risk Application

Credit risk arises as a result of the failure of the loans given by banks to the customers to fulfill their obligations at the end of the specified term. Technological advances allow the use of machine learning methods in various sectors. These methods aim to facilitate the identification of customers at risk with the system adapted to the creditworthiness processes of banks. For this purpose, in order to make the most appropriate evaluation in the lending process of banks, re-sampling techniques to eliminate the problem of class imbalance encountered in unbalanced data sets were made balanced and their effects on machine learning were investigated. During the implementation phase, German, Australian and HMEQ credit data sets were used. Different machine learning classification methods such as Logistic Regression (LR), K-Narest Neighbor (KNN), Naive Bayes (NB), Support Vector Machines (SVM), Multilayer Perceptron (MLP), Decision Trees (DT), Random Forests (RF), Gradient Boosting Decision Trees (GBDT), Extremely Randomized Trees, Hard and Soft Voting were used to detect risky customers. The problem of class imbalance was balanced with resampling and hybrid techniques such as Random Oversampling (ROS), Random Undersampling (RUS), Balanced Bagging Classifier (BBC), SMOTE-Tomek Links and SMOTE-ENN. In this context, the performances of three different data sets were examined in four different scenarios. As a result of the study, the hybrid method, in which oversampling and undersampling methods are used together for the class balancing problem, showed the best classification performance among machine learning techniques.

Robust two-stage instance-level cost-sensitive learning method for class imbalance problem

The class imbalance problem is a pervasive challenge in various applications, prompting researchers to design specialized loss functions for instance-level cost-sensitive learning models. However, the performance deteriorates when datasets are contaminated by label noise, especially in scenarios with uneven noise distribution. Additionally, specific hard-to-classify samples can further impede performance. Addressing these issues and establishing a robust instance-level cost-sensitive learning framework remains a formidable challenge. In this paper, we propose a robust Two-Stage instance-level cost-sensitive learning method with the Bounded Quadratic Type Squared Error (BQTSE) loss function, termed TSBQT. The asymmetric and bounded nature of the BQTSE loss function not only allows for distinct cost assignments to majority and minority instances, but also enhances the robustness to uneven noise. Moreover, our two-stage learning strategy prioritizes hard-to-classify instances by considering their classification difficulty. We solve TSBQT using the alternating direction method of multipliers and gradient descent algorithm. The generalization capability of TSBQT is analyzed with the Rademacher complexity theory. Extensive experiments validate the effectiveness of TSBQT.

CARBO: Clustering and rotation based oversampling for class imbalance learning

Class imbalance of a data set is a crucial problem in machine learning where one class significantly outnumbers others. In such a data set, classification is a troublesome task for the standard classification algorithms, leading to bias towards the majority class. Different methods have been developed so far, such as oversampling, undersampling, and cost-sensitive learning, to deal with class imbalance circumstances. Among these techniques, oversampling technique does not suffer from the information loss and critical cost selection challenges. However, appropriate synthetic sample generation can be challenging and vulnerable to privacy leakage. This research proposed an oversampling technique, called CARBO, using threshold-based geometric rotation and majority class influenced clustering. Unlike the existing resampling approaches to class imbalance problem, we contribute to consider the data privacy and optimal sample generation together for effective oversampling. The performance of CARBO is evaluated using 44 benchmark imbalanced data set. The empirical analysis elucidates that CARBO can make boosting-based C4.5 ensemble classifiers perform higher for 73% of the data set than six state-of-the-art approaches. In addition, the theoretical compatibility analysis of CARBO demonstrates its robustness.

Multiscale Feature Fusion Method for Liver Cirrhosis Classification

ABSTRACTLiver cirrhosis is one of the most common liver diseases in the world, posing a threat to people's daily lives. In advanced stages, cirrhosis can lead to severe symptoms and complications, making early detection and treatment crucial. This study aims to address this critical healthcare challenge by improving the accuracy of liver cirrhosis classification using ultrasound imaging, thereby assisting medical professionals in early diagnosis and intervention. This article proposes a new multiscale feature fusion network model (MSFNet), which uses the feature extraction module to capture multiscale features from ultrasound images. This approach enables the neural network to utilize richer information to accurately classify the stage of cirrhosis. In addition, a new loss function is proposed to solve the class imbalance problem in medical datasets, which makes the model pay more attention to the samples that are difficult to classify and improves the performance of the model. The effectiveness of the proposed MSFNet was evaluated using ultrasound images from 61 subjects. Experimental results demonstrate that our method achieves high classification accuracy, with 98.08% on convex array datasets and 97.60% on linear array datasets. Our proposed method can classify early, middle, and late cirrhosis very accurately. It provides valuable insights for the clinical treatment of liver cirrhosis and may be helpful for the rehabilitation of patients.

Lazy Multi-Label Classification algorithms based on Non-Parametric Predictive Inference

Multi-Label Classification (MLC) extends standard classification in the sense that an instance might belong to multiple labels simultaneously. Many lazy approaches to MLC have been proposed so far. The majority of them, to classify an instance, use statistical estimators from the neighboring instances based on classical probability theory. In this work, we propose lazy algorithms for MLC that employ the Non-Parametric Predictive Inference Model (NPI-M) for the statistical estimators based on the neighboring instances. It is shown that our proposed lazy MLC algorithms are more suitable to tackle the class-imbalance problem that usually arises in MLC, especially when data contain label noise. This issue is corroborated via an exhaustive experimental analysis.

EKMGS: A HYBRID CLASS BALANCING METHOD FOR MEDICAL DATA PROCESSING

The field of medicine is witnessing rapid development of AI, highlighting the importance of proper data processing. However, when working with medical data, there is a problem of class imbalance, where the amount of data about healthy patients significantly exceeds the amount of data about sick ones. This leads to incorrect classification of the minority class, resulting in inefficient operation of machine learning algorithms. In this study, a hybrid method was developed to address the problem of class imbalance, combining oversampling (GenSMOTE) and undersampling (ENN) algorithms. GenSMOTE used frequency oversampling optimization based on a genetic algorithm, selecting the optimal value using a fitness function. The next stage implemented an ensemble method based on stacking, consisting of three base (k-NN, SVM, LR) and one meta-model (Decision Tree). The hyperparameters of the meta-model were optimized using the GridSearchCV algorithm. During the study, datasets on diabetes, liver diseases, and brain glioma were used. The developed hybrid class balancing method significantly improved the quality of the model: the F1-score increased by 10-75%, and accuracy by 5-30%. Each stage of the hybrid algorithm was visualized using a nonlinear UMAP algorithm. The ensemble method based on stacking, in combination with the hybrid class balancing method, demonstrated high efficiency in solving classification tasks in medicine. This approach can be applied for diagnosing various diseases, which will increase the accuracy and reliability of forecasts. It is planned to expand the application of this approach to large volumes of data and improve the oversampling algorithm using additional capabilities of the genetic algorithm.

Manifold neighboring envelope sample generation mechanism for imbalanced ensemble classification

For existing imbalanced ensemble (IE) methods, the sample subsets are constructed from the same dataset, which usually suffer from low quality (diversity and separability) of the subsets, so a manifold neighboring envelope sample generation mechanism (MNESG) and an imbalanced ensemble algorithm based on the mechanism (MNESG-IE) are proposed to solve this problem. First, for the original balanced subsets (OBS), a manifold neighboring sample envelope projection mechanism (MNSEP) is designed to mine the local correlation information between the samples and their nearest neighbors in the subsets. Second, the fuzzy c-means clustering (FCM) is used to further mine the global correlation information among similar samples in the subsets. Third, the sample distribution consistency preservation mechanism (SDCPM) is designed to enhance the consistency of the sample distribution before and after clustering. To better reduce the three accumulated losses above, the three steps are conducted simultaneously, thereby realizing the MNESG, which can transform the OBS into two new types of high quality envelope sample subsets − neighboring envelope sample (NES) subsets and neighboring cluster envelope sample (NCES) subsets. Finally, base classifiers are trained on the NES subsets and NCES subsets, and then fused by a two-dimensional sparse fusion mechanism (2D-SFM). Various representative IE algorithms on over thirty benchmark datasets are considered for verification. The results show that compared with the state-of-the-art IE algorithms, MNESG-IE achieves 17.79%, 17.90%, 23.61%, 18.08% improvement in terms of ACC, AUC, F-M and G-M, respectively. The major originality of the paper is: (a) proposing the MNSEP to mine the local correlation information for improving the quality of the subsets; (b) proposing the MNESG to generate high quality subsets by mining local and global correlation information simultaneously; and (c)forming an IE algorithm to better solve the imbalanced classification problem.

Class-overlap detection based on heterogeneous clustering ensemble for multi-class imbalance problem

The class imbalance problem is one of the main challenges that hinders classifiers from identifying unknown instances. When class distribution imbalance and class overlap coexist in the same data set, the recognition of minority instances by traditional classifiers will be seriously affected. In the binary classification problem, the relationship between the majority class and the minority class is clear, and the overlap problem between classes can be well solved by removing the majority instances in the class-overlap area. However, for multi-class imbalance problems, the relationships between classes will become complex. Clustering techniques are effective methods for identifying overlapping instances; however, a single clustering method cannot effectively identify clusters of different shapes. Therefore, in this paper, a Heterogeneous Clustering Ensemble learning method for Multiple Class-overlap Detection (HCE-MCD) is proposed for multi-class imbalance problems. This method uses a genetic algorithm to select a combination of heterogeneous clustering techniques with the best fitness function from a pool of clustering algorithms and uses majority voting to fuse the results of the clustering algorithm. Furthermore, instances located near the decision boundary are more likely to be overlapping instances. Therefore, according to the clustering results of the heterogeneous clustering ensemble method, we can identify the overlapping instances existing in the data set, thereby eliminating the class overlap problem in the multi-class dataset. Experimental results on 19 open-access datasets show that our proposed method outperforms or partially outperforms state-of-the-art research protocols.

ImFTP: Deep imbalance learning via fuzzy transition and prototypical learning

Although many methods have been proposed for tackling the class-imbalance problem, they still suffer from the insufficient feature representative capability and the overfitting problem. This paper proposes a new deep imbalance learning approach based on the Fuzzy set Theory and the Prototypical learning mechanism, abbreviated as imFTP for short, which consists of an adaptive smooth sampling module, a self-learnable prototypical learning module, and a fuzzy transition module. The adaptive smooth sampling module adaptively adjusts the sampling frequency of different classes to ensure their adequate opportunity to participate in the training process, which can mitigate the overfitting problem. The self-learnable prototypical learning module devises a clustering distance based Softmax cross-entropy loss and an intra-class clustering loss to improve the feature representation and discrimination capability of the model. The fuzzy transition module utilizes the fuzzy set theory to transform sample features effectively, which further enhances the feature representation capability of the model, meanwhile alleviates the overfitting problem. Experimental results on 15 benchmark datasets demonstrate that our method outperforms the best competitor by more than 3% in terms of the Macro-F1 metric, which is very significant.

Distance mapping overlap complexity metric for class-imbalance problems

Data complexity for class-imbalance problems is a hot topic in the field of data mining. Classical data complexity measures use techniques such as kNN, which computes the nearest neighbors of each instance. However, the kNN-based method to obtain the nearest neighbors of all instances is an NP-hard problem, which is not conducive to large-scale data complexity calculations. In order to solve this problem, from the global and local perspectives, four distance map complexity metric methods are proposed, and the complexity values is called the distance map overlap index (DMOI). Firstly, we use Mahalanobis distance or standardized Euclidean distance to calculate the distance of the instance to the center point of the data set, and arrange by the distance value of each instance. Then, according to the sequential label vector, iteratively searches the number of map cut-points for each class. Finally, according to the number of map cut-points of different classes, the DMOI of the data set is calculated. Experiments on 50 class-imbalanced datasets show that the DMOI method outperforms state-of-the-art complexity measures for class-imbalanced problems. Although there is still a gap between DMOI and ONB for rule-based or tree-based classifiers, the use of Pearson correlation coefficient shows that DMOI can effectively approximate ONB, and there is a strong positive correlation between them.

An Improved YOLOv8 Model for Lotus Seedpod Instance Segmentation in the Lotus Pond Environment

Lotus seedpod maturity detection and segmentation in pond environments play a significant role in yield prediction and picking pose estimation for lotus seedpods. However, it is a great challenge to accurately detect and segment lotus seedpods due to insignificant phenotypic differences between the adjacent maturity, changing illumination, overlap, and occlusion of lotus seedpods. The existing research pays attention to lotus seedpod detection while ignoring maturity detection and segmentation problems. Therefore, a semantic segmentation dataset of lotus seedpods was created, where a copy-and-paste data augmentation tool was employed to eliminate the class-imbalanced problem and improve model generalization ability. Afterwards, an improved YOLOv8-seg model was proposed to detect and segment the maturity of lotus seedpods. In the model, the convolutional block attention module (CBAM) was embedded in the neck network to extract distinguished features of different maturity stages with negligible computation cost. Wise-Intersection over Union (WIoU) regression loss function was adopted to refine the regression inference bias and improve the bounding box prediction accuracy. The experimental results showed that the proposed YOLOv8-seg model provides an effective method for “ripe” and “overripe” lotus seedpod detection and instance segmentation, where the mean average precision of segmentation mask (mAPmask) reaches 97.4% and 98.6%, respectively. In addition, the improved YOLOv8-seg exhibits high robustness and adaptability to complex illumination in a challenging environment. Comparative experiments were conducted using the proposed YOLOv8-seg and other state-of-the-art instance segmentation methods. The results showed that the improved model is superior to the Mask R-CNN and YOLACT models, with recall, precision, mAPbox and mAPmask being 96.5%, 94.3%, 97.8%, and 98%, respectively. The average running time and weight size of the proposed model are 25.9 ms and 7.4 M, respectively. The proposed model obtained the highest mAP for lotus seedpod maturity detection and segmentation while maintaining an appropriate model size and speed. Furthermore, based on the obtained segmentation model, 3D visualization of the lotus pond scene is performed, and cloud point of lotus seedpods is generated, which provides a theoretical foundation for robot harvesting in the lotus pond.

Imbalanced and missing multi-label data learning with global and local structure

Label missing and class imbalance problems are two hot research topics in machine learning, and they have been impeding the improvement of model performance, especially in the multi-label learning. Although some existing methods have proven to be effective, they are suitable for only one case. How to effectively address above two issues simultaneously is a challenging problem. In this paper, we propose a novel model named Imbalanced and Missing multi-Label data learning with Global and Local structure (IMLGL) to address the aforementioned challenge. There are following three advantages. At the empirical risk level, we introduce the label correlation matrix C into the loss function and devise a dynamic weighting method to address the aforementioned challenge. At the data level, we analyze the structural characteristics of the data, and introduce local low-rank and global high-rank term to enhance the generalization performance of the model. At the label level, a smoothing term is also introduced for learning the constraint classifier coefficient matrix W. Our method utilizes alternative optimization technique and alternating minimization method for solving. Extensive experiments on six datasets demonstrate the competitiveness of our approach.

Random Forest and CatBoost with Handling Imbalanced Class for Detection of Risk Factors Anemia in Children (5-12 Years)

The prevalence of anemia in children (5-12 years) remains a public health issue in Indonesia. Early detection and control of risk factors are crucial for prevention. Machine learning models can be employed to address this problem. One practical approach is using ensemble learning models. However, it is expected to encounter imbalanced class problems when analyzing health data. Therefore, this study aims to perform classification modeling using two ensemble learning models: Random Forest (RF) and CatBoost. The proposed methods for handling imbalanced class issues include Random Over Sampling, SMOTE, G-SMOTE, Random Under Sampling, Instance Hardness Threshold (IHT), and SMOTE-ENN. Additionally, SHAP is used to explain the best-performing model based on Shapley values. The research findings indicate that the ensemble learning model using the CatBoost algorithm with G-SMOTE data handling produces the best performance compared to other methods. Based on the average performance metrics from 100 replicate validation, the CatBoost G-SMOTE model produces a sensitivity of 0.7104, specificity of 0.7043, G-Mean of 0.7067, and AUC of 0.7844. Handling the imbalance class problem using the G-SMOTE method effectively increases the sensitivity value in the two proposed ensemble learning models. Meanwhile, the SMOTE-ENN method produces effective G-Mean values for the Random Forest (RF) algorithms. Based on Shapley's value, the features with the highest contribution to predicting anemia in children (5-12 years) are ferritin, vitamin A, consumption of vegetables, diagnosed pneumonia, zinc, calcium total, and consumption of soft or carbonated drinks.

Toward Interpretable Cell Image Representation and Abnormality Scoring for Cervical Cancer Screening Using Pap Smears.

Screening is critical for prevention and early detection of cervical cancer but it is time-consuming and laborious. Supervised deep convolutional neural networks have been developed to automate pap smear screening and the results are promising. However, the interest in using only normal samples to train deep neural networks has increased owing to the class imbalance problems and high-labeling costs that are both prevalent in healthcare. In this study, we introduce a method to learn explainable deep cervical cell representations for pap smear cytology images based on one-class classification using variational autoencoders. Findings demonstrate that a score can be calculated for cell abnormality without training models with abnormal samples, and we localize abnormality to interpret our results with a novel metric based on absolute difference in cross-entropy in agglomerative clustering. The best model that discriminates squamous cell carcinoma (SCC) from normals gives 0.908±0.003 area under operating characteristic curve (AUC) and one that discriminates high-grade epithelial lesion (HSIL) 0.920±0.002 AUC. Compared to other clustering methods, our method enhances the V-measure and yields higher homogeneity scores, which more effectively isolate different abnormality regions, aiding in the interpretation of our results. Evaluation using an external dataset shows that our model can discriminate abnormality without the need for additional training of deep models.

CPS-3WS: A critical pattern supported three-way sampling method for classifying class-overlapped imbalanced data

Class-imbalance problem widely exists in real applications ranging from medial diagnosis to economic fraud detection, etc. As one of the mainstream techniques in dealing with imbalanced data, SMOTE (Synthetic Minority Over-sampling TEchnique) and its extensions mainly rebalance the datasets via generation of observations in specific regions with various adapted strategies. Many of them do not consider the cost of role assignment of samples, and the intractable data complexity (overlap, small disjuncts, etc.) poses additional challenges to them. This paper proposes a critical pattern supported three-way sampling method (CPS-3WS) for classifying class-overlapped imbalanced data, introducing the philosophy of thinking in threes to effective classification in imbalanced learning. Specifically, CPS-3WS uses a three-way sample partition strategy with the Bayes posterior probability by dividing majority and minority classes into three disjoint subsets: risky, critical and safe patterns. CPS-3WS conducts a three-way hybrid sampling through (i) evaluating the risky majority pattern to be eliminated and (ii) selecting critical minority pattern to synthesize new samples under local information constraint. Extensive experiments on 42 UCI benchmark datasets demonstrate the superiority of the proposed CPS-3WS compared with 11 data-level methods. The source code of CPS-3WS is available at https://github.com/ytyancp/CPS-3WS.

A class-imbalanced hybrid learning strategy based on Raman spectroscopy of serum samples for the diagnosis of hepatitis B, hepatitis A, and thyroid dysfunction

Computer-aided vibrational spectroscopy detection technology has achieved promising results in the field of early disease diagnosis. Yet limited by factors such as the number of actual samples and the cost of spectral acquisition in clinical medicine, the data available for model training are insufficient, and the amount of data varies greatly between different diseases, which constrain the performance optimization and enhancement of the diagnostic model. In this study, vibrational spectroscopy data of three common diseases are selected as research objects, and experimental research is conducted around the class imbalance situation that exists in medical data. When dealing with the challenge of class imbalance in medical vibrational spectroscopy research, it no longer relies on some kind of independent and single method, but considers the combined effect of multiple strategies. SVM, K-Nearest Neighbor (KNN), and Decision Tree (DT) are used as baseline comparison models on Raman spectroscopy medical datasets with different imbalance rates. The performance of the three strategies, Ensemble Learning, Feature Extraction, and Resampling, is verified on the class imbalance dataset by G-mean and AUC metrics, respectively. The results show that all the above three methods mitigate the negative impact caused by unbalanced learning. Based on this, we propose a hybrid ensemble classifier (HEC) that integrates resampling, feature extraction, and ensemble learning to verify the effectiveness of the hybrid learning strategy in solving the class imbalance problem. The G-mean and AUC values of the HEC method are 82.7 % and 83.12 % for the HBV dataset, is 2.02 % and 1.98 % higher than the optimal strategy; 83.62 % and 83.76 % for the HCV dataset, is 9.79 % and 8.47 % higher than the optimal strategy; while for the thyroid dysfunction dataset are 77.56 % and 77.85 %, is 6.92 % and 6.36 % higher than that of the optimal strategy, respectively. The experimental results show that the G-mean and AUC metrics of the HEC method are higher than those of the baseline classifier as well as the optimal combination using separate strategies. It can be seen that the HEC method can effectively counteract the unfavorable effects of imbalance learning and is expected to be applied to a wider range of imbalance scenarios.

Underwater object detection in noisy imbalanced datasets

Class imbalance occurs in the datasets with a disproportionate ratio of observations. The class imbalance problem drives the detection and classification systems to be more biased towards the over-represented classes while the under-represented classes may not receive sufficient learning. Previous works often deploy distribution based re-balancing approaches to address this problem. However, these established techniques do not work properly for underwater object detection where label noise commonly exists. In our experiments, we observe that the imbalanced detection problem may be caused by imbalance data distributions or label noise. To deal with these challenges, we first propose a noise removal (NR) algorithm to remove label noise in the datasets, and then propose a factor-agnostic gradient re-weighting algorithm (FAGR) to address the imbalanced detection problem. FAGR provides a rebalanced gradient to each class, which encourages the detection network to treat all the classes equally whilst minimising the detection discrepancy. Our proposed NR+FAGR framework achieves state-of-the-art (SOAT) performance on three underwater object datasets due to its high capacity in handling the class imbalance and noise issues. The source code will be made available at: https://github.com/IanDragon.