Imbalanced Data Research Articles

Comparative Analysis of Cross-Validation Techniques: LOOCV, K-folds Cross-Validation, and Repeated K-folds Cross-Validation in Machine Learning Models

Effective model evaluation is crucial for robust machine learning, and cross-validation techniques play a significant role. This study compares Repeated k-folds Cross Validation, k-folds Cross Validation, and Leave-One-Out Cross Validation (LOOCV) on imbalanced and balanced datasets across four models: Support Vector Machine (SVM), K-Nearest Neighbors (K-NN), Random Forest (RF), and Bagging, both with and without parameter tuning. On imbalanced data without parameter tuning, Repeated k-folds cross-validation demonstrated strong performance for SVM with a sensitivity of 0.541 and balanced accuracy of 0.764. K-folds Cross Validation showed a higher sensitivity of 0.784 for RF and a balanced accuracy of 0.884. In contrast, LOOCV achieved notable sensitivity for RF and Bagging at 0.787 and 0.784, respectively, but at the cost of lower precision and higher variance, as detailed in Table 1. When parameter tuning was applied to balanced data, the performance metrics improved. Sensitivity for SVM reached 0.893 with LOOCV and balanced accuracy for Bagging increased to 0.895. Stratified k-folds provided enhanced precision and F1-Score for SVM and RF. Notably, processing times varied significantly, with k-folds being the most efficient with SVM taking 21.480 seconds and Repeated k-folds showing higher computational demands where RF took approximately 1986.570 seconds in model processing, as shown in Table 4. This analysis underscores that while k-folds and repeated k-folds are generally efficient, LOOCV and balanced approaches offer enhanced accuracy for specific models but require greater computational resources. The choice of cross-validation technique should thus be tailored to the dataset characteristics and computational constraints to ensure optimal model evaluation.

SS4CTR: a semi-supervised framework for enhancing click-through rate prediction in sparse and imbalanced data

SS4CTR: a semi-supervised framework for enhancing click-through rate prediction in sparse and imbalanced data

Semi-Supervised Encrypted Malicious Traffic Detection Based on Multimodal Traffic Characteristics.

The exponential growth of encrypted network traffic poses significant challenges for detecting malicious activities online. The scale of emerging malicious traffic is significantly smaller than that of normal traffic, and the imbalanced data distribution poses challenges for detection. However, most existing methods rely on single-category features for classification, which struggle to detect covert malicious traffic behaviors. In this paper, we introduce a novel semi-supervised approach to identify malicious traffic by leveraging multimodal traffic characteristics. By integrating the sequence and topological information inherent in the traffic, we achieve a multifaceted representation of encrypted traffic. We design two independent neural networks to learn the corresponding sequence and topological features from the traffic. This dual-feature extraction enhances the model's robustness in detecting anomalies within encrypted traffic. The model is trained using a joint strategy that minimizes both the reconstruction error from the autoencoder and the classification loss, allowing it to effectively utilize limited labeled data alongside a large amount of unlabeled data. A confidence-estimation module enhances the classifier's ability to detect unknown attacks. Finally, our method is evaluated on two benchmark datasets, UNSW-NB15 and CICIDS2017, under various scenarios, including different training set label ratios and the presence of unknown attacks. Our model outperforms other models by 3.49% and 5.69% in F1 score at labeling rates of 1% and 0.1%, respectively.

Ensembles of decision trees and gradient-based learning for employee turnover rate prediction

Employee turnover has a negative impact on business profitability. To tackle this issue, we can utilize computational advancements to forecast attrition and minimize expenses. We employed an HR Analytics dataset to investigate the feasibility of using these predictive models in decision support systems. We developed an ensemble of gradient-based decision trees that accurately predicted employee turnover and performed better than other sophisticated techniques. This approach demonstrates exceptional performance in handling structured and imbalanced data, effectively capturing intricate patterns. Gradient-based decision trees provide scalable solutions that effectively balance predictive accuracy and computational efficiency, making them well-suited for strategic business analysis. The importance of our findings lies in their ability to offer dependable insights for making well-informed decisions in business settings.

Bayesian optimized deep Q-network for diagnosing mine ventilation systems windage alteration fault targeting imbalanced data

Fault diagnosis of mine ventilation system is of great significance for mine safety production. Traditional machine learning algorithms have been widely applied in the field of mine ventilation systems windage alteration faults (WAFs) diagnosis, but these algorithms have poor intelligence and weak generalization ability. Therefore, this research constructs a WAFs diagnosis model (WAFs-BODQN) based on Bayesian optimized (BO) deep Q-network (DQN). Meanwhile, in order to solve the problem of poor performance of intelligent models when the dataset is imbalanced, a reinforcement learning reward function (KMeans-SDF) was designed, which combines the KMeans++ algorithm and spatial distance function (SDF). The WAFs-BODQN model with KMeans-SDF reward function has fault location diagnosis accuracies of 98.75 %, 97.50 %, and 95.00 % for mild, moderate, and extreme imbalanced datasets UB1, UB2, and UB3, respectively. This effectively avoids the "eccentricity" phenomenon of the intelligent model on majority class and achieves accurate prediction of fault locations. The accuracy of fault location diagnosis using the WAFs-BODQN model in simulation experiments and on-site empirical applications is higher than 95 %, demonstrating the intelligence, generalization, and feasibility of WAFs-BODQN model in the field of WAFs diagnosis in mine ventilation systems. By comparing the results with traditional machine learning and deep learning fault diagnosis models, it has been proven that the WAFs-BODQN model has advantages in intelligence and generalization ability. It is expected to establish a universal architecture for diagnosing WAFs, providing theoretical guidance and technical support for achieving intelligent mine ventilation.

Feature Bagging with Nested Rotations (FBNR) for anomaly detection in multivariate time series

Detecting anomalies in multivariate time series poses a significant challenge across various domains. The infrequent occurrence of anomalies in real-world data, as well as the lack of a large number of annotated samples, makes it a complex task for classification algorithms. Deep Neural Network approaches, based on Long Short-Term Memory (LSTMs), Autoencoders, and Variational Autoencoders (VAEs), among others, prove effective with handling imbalanced data. However, the same does not follow when such algorithms are applied on multivariate time-series, as their performance degrades significantly. Our main hypothesis is that the above is due to anomalies stemming from a small subset of the feature set. To mitigate the above issues in the multivariate setting, we propose forming an ensemble of base models by combining different feature selection and transformation techniques. The proposed processing pipeline includes applying a Feature Bagging techniques on multiple individual models, which considers separate feature subsets for each specific model. These subsets are then partitioned and transformed using multiple nested rotations derived from Principal Component Analysis (PCA). This approach aims to identify anomalies that arise from only a small portion of the feature set while also introduces diversity by transforming the subspaces. Each model provides an anomaly score, which are then aggregated, via an unsupervised decision fusion model. A semi-supervised fusion model was also explored, in which a Logistic Regressor was applied on the individual model outputs. The proposed methodology is evaluated on the Skoltech Anomaly Benchmark (SKAB), containing multivariate time series related to water flow in a closed circuit, as well as the Server Machine Dataset (SMD), which was collected from a large Internet company. The experimental results reveal that the proposed ensemble technique surpasses state-of-the-art algorithms. The unsupervised approach demonstrated a performance improvement of 2% for SKAB and 3% for SMD, compared to the baseline models. In the semi-supervised approach, the proposed method achieved a minimum of 10% improvement in terms of anomaly detection accuracy.

NNEnsLeG: A novel approach for e-commerce payment fraud detection using ensemble learning and neural networks

The proliferation of fraud in online shopping has accompanied the development of e-commerce, leading to substantial economic losses, and affecting consumer trust in online shopping. However, few studies have focused on fraud detection in e-commerce due to its diversity and dynamism. In this work, we conduct a feature set specifically for e-commerce payment fraud, around transactions, user behavior, and account relevance. We propose a novel comprehensive model called Neural Network Based Ensemble Learning with Generation (NNEnsLeG) for fraud detection. In this model, ensemble learning, data generation, and parameter-passing are designed to cope with extreme data imbalance, overfitting, and simulating the dynamics of fraud patterns. We evaluate the model performance in e-commerce payment fraud detection with >310,000 pieces of e-commerce account data. Then we verify the effectiveness of the model design and feature engineering through ablation experiments, and validate the generalization ability of the model in other payment fraud scenarios. The experimental results show that NNEnsLeG outperforms all the benchmarks and proves the effectiveness of generative data and parameter-passing design, presenting the practical application of the NNEnsLeG model in e-commerce payment fraud detection.

Imbalanced Data Parameter Optimization of Convolutional Neural Networks Based on Analysis of Variance

Classifying imbalanced data is important due to the significant practical value of accurately categorizing minority class samples, garnering considerable interest in many scientific domains. This study primarily uses analysis of variance (ANOVA) to investigate the main and interaction effects of different parameters on imbalanced data, aiming to optimize convolutional neural network (CNN) parameters to improve minority class sample recognition. The CIFAR-10 and Fashion-MNIST datasets are used to extract samples with imbalance ratios of 25:1, 15:1, and 1:1. To thoroughly assess model performance on imbalanced data, we employ various evaluation metrics, such as accuracy, recall, F1 score, P-mean, and G-mean. In highly imbalanced datasets, optimizing the learning rate significantly affects all performance metrics. The interaction between the learning rate and kernel size significantly impacts minority class samples in moderately imbalanced datasets. Through parameter optimization, the accuracy of the CNN model on the 25:1 highly imbalanced CIFAR-10 and Fashion-MNIST datasets improves by 14.20% and 5.19% compared to the default model and by 8.21% and 3.87% compared to the undersampling model, respectively, while also enhancing other evaluation metrics for minority classes.

Learning from Imbalanced Data: Integration of Advanced Resampling Techniques and Machine Learning Models for Enhanced Cancer Diagnosis and Prognosis

Background/Objectives: This study aims to evaluate the performance of various classification algorithms and resampling methods across multiple diagnostic and prognostic cancer datasets, addressing the challenges of class imbalance. Methods: A total of five datasets were analyzed, including three diagnostic datasets (Wisconsin Breast Cancer Database, Cancer Prediction Dataset, Lung Cancer Detection Dataset) and two prognostic datasets (Seer Breast Cancer Dataset, Differentiated Thyroid Cancer Recurrence Dataset). Nineteen resampling methods from three categories were employed, and ten classifiers from four distinct categories were utilized for comparison. Results: The results demonstrated that hybrid sampling methods, particularly SMOTEENN, achieved the highest mean performance at 98.19%, followed by IHT (97.20%) and RENN (96.48%). In terms of classifiers, Random Forest showed the best performance with a mean value of 94.69%, with Balanced Random Forest and XGBoost following closely. The baseline method (no resampling) yielded a significantly lower performance of 91.33%, highlighting the effectiveness of resampling techniques in improving model outcomes. Conclusions: This research underscores the importance of resampling methods in enhancing classification performance on imbalanced datasets, providing valuable insights for researchers and healthcare professionals. The findings serve as a foundation for future studies aimed at integrating machine learning techniques in cancer diagnosis and prognosis, with recommendations for further research on hybrid models and clinical applications.

Automatic Acne Severity Grading with a Small and Imbalanced Data Set of Low-Resolution Images.

Developing automatic acne vulgaris grading systems based on machine learning is an expensive endeavor in terms of data acquisition. A machine learning practitioner will need to gather high-resolution pictures from a considerable number of different patients, with a well-balanced distribution between acne severity grades and potentially very tedious labeling. We developed a deep learning model to grade acne severity with respect to the Investigator's Global Assessment (IGA) scale that can be trained on low-resolution images, with pictures from a small number of different patients, a strongly imbalanced severity grade distribution and minimal labeling. A total of 1374 triplets of images (frontal and lateral views) from 391 different patients suffering from acne labeled with the IGA severity grade by an expert dermatologist were used to train and validate a deep learning model that predicts the IGA severity grade. On the test set we obtained 66.67% accuracy with an equivalent performance for all grades despite the highly imbalanced severity grade distribution of our database. Importantly, we obtained performance on par with more tedious methods in terms of data acquisition which have the same simple labeling as ours but require either a more balanced severity grade distribution or large numbers of high-resolution images. Our deep learning model demonstrated promising accuracy despite the limited data set on which it was trained, indicating its potential for further development both as an assistance tool for medical practitioners and as a way to provide patients with an immediately available and standardized acne grading tool. chinadrugtrials.org.cn identifier CTR20211314.

Multi-Class Imbalanced Data Handling with Concept Drift in Fog Computing: A Taxonomy, Review, and Future Directions

A network of actual physical objects or “IoT components” linked to the internet and equipped with sensors, electronics, software, and network connectivity is known as the Internet of Things (IoT). This ability of the IoT components to gather and share data is made possible by this network connectivity. Many IoT devices are currently operating, which generate a lot of data. When these IoT devices started collecting data, the cloud was the only place to analyze, filter, pre-process, and aggregate it. However, when it comes to IoT, the cloud has restrictions regarding latency and a more centralized method of distributing programs. A new form of computing called Fog computing has been proposed to address the shortcomings of current cloud computing. In an IoT context, sensors regularly communicate signal information, and edge devices process the data obtained from these sensors using Fog computing. The sensors’ internal or external problems, security breaches, or the integration of heterogeneous equipment contribute to the imbalanced data, i.e., comparatively speaking, one class has more instances than the other. As a result of this data, the pattern extraction is imbalanced . Recent attempts have concentrated heavily on binary-class imbalanced concerns with exactly two classes. However, the classification of multi-class imbalanced data is an issue that needs to be fixed in Fog computing, even if it is widespread in other fields, including text categorization, human activity detection, and medical diagnosis. The study intends to deal with this problem. It presents a systematic, thorough, and in-depth comparative analysis of several binary-class and multi-class imbalanced data handling strategies for batch and streaming data in IoT networks and Fog computing. There are five major objectives in this study. First, reviewing the Fog computing concept. Second, outlining the optimization metric used in Fog computing. Third, focusing on binary and multi-class batch data handling for IoT networks and Fog computing. Fourth, reviewing and comparing the current imbalanced data handling methodologies for multi-class data streams. Fifth, explaining how to cope with the concept drift, including novel and recurring classes, targeted optimization measures, and evaluation tools. Finally, the best performance metrics and tools for concept drift, binary-class (batch and stream) data, and multi-class (batch and stream) data are highlighted.

Imbalanced image classification algorithm based on fine-grained analysis

Fine-grained attribute analysis and data imbalance have always been research hotspots in the field of computer vision. Due to the complexity and diversity of fine-grained attribute images, traditional image classification methods have shortcomings in paying attention to fine-grained attributes of images and perform poorly when dealing with imbalanced data sets. To overcome these problems, this study proposes a fine-grained image threshold classification algorithm based on deep metric learning. By introducing a metric learning method, the focus on fine-grained attributes of images is enhanced. At the same time, by applying pairwise loss and proxy loss, the classification accuracy of the model is improved and the model convergence speed is accelerated. In order to deal with the problem of data imbalance, a classifier based on threshold analysis is designed. The classifier uses threshold analysis technology to achieve multi-level classification of fine-grained images, thereby improving the problem of low classification accuracy of a few categories in imbalanced data sets. Experimental results show that the proposed fine-grained image threshold classification algorithm based on deep metric learning is significantly superior to other methods in terms of classification accuracy.

A review of deep learning approaches for multimodal image segmentation of liver cancer.

This review examines the recent developments in deep learning (DL) techniques applied to multimodal fusion image segmentation for liver cancer. Hepatocellular carcinoma is a highly dangerous malignant tumor that requires accurate image segmentation for effective treatment and disease monitoring. Multimodal image fusion has the potential to offer more comprehensive information and more precise segmentation, and DL techniques have achieved remarkable progress in this domain. This paper starts with an introduction to liver cancer, then explains the preprocessing and fusion methods for multimodal images, then explores the application of DL methods in this area. Various DL architectures such as convolutional neural networks (CNN) and U-Net are discussed and their benefits in multimodal image fusion segmentation. Furthermore, various evaluation metrics and datasets currently used to measure the performance of segmentation models are reviewed. While reviewing the progress, the challenges of current research, such as data imbalance, model generalization, and model interpretability, are emphasized and future research directions are suggested. The application of DL in multimodal image segmentation for liver cancer is transforming the field of medical imaging and is expected to further enhance the accuracy and efficiency of clinical decision making. This review provides useful insights and guidance for medical practitioners.

A novel reinforcement learning agent for rotating machinery fault diagnosis with data augmentation

The problem of data imbalance and limited sample poses a prominent challenge to fault diagnosis in industry, especially in rotating machinery fault diagnosis due to the constraints of sampling and labeling. Reinforcement learning is known for its auto-optimized learning ability but it is severely limited by the quality and quantity of data it is able to collect. This paper applies the Equilibrium Deep Q-Network (Equilibrium-DQN) based agent with the Variational Autoencoder with Wasserstein Generative Adversarial Network and Gradient Penalty (VAE-WGAN-GP) for rotating machinery fault diagnosis. The Equilibrium-DQN possesses the property of leveraging multiple target networks to improve training stability and accuracy. The VAE-WGAN-GP excels in resolving data scarcity problems in fault diagnosis, especially in enriching labeled data with more typical samples while keeping the original data features. Herein, a novel framework is presented that integrates both data augmentation techniques with reinforcement learning agent for fault diagnosis. The proposed framework improves the weakness of insufficient samples in fault diagnosis and reveals its potential in complex industrial applications. The experiments, both on the test bench dataset and on the real locomotive dataset, confirm the advantage of the proposed framework.

On the use of Change History Data to Enhance Class Change-Proneness Prediction Models

As software evolves, new artifacts are created, modified, or removed. One of these main artifacts generated in the development of object-oriented software is the class. Classes have a very dynamic life cycle that can result in additional costs to the project. One way to mitigate this is to detect, in the early stages of the development, classes that are prone to change. Some approaches in the literature adopt Machine Learning (ML) algorithms to predict the change-proneness of a class. However, most of these approaches do not consider the temporal dependency between training instances, i.e., they consider that the instances are independent. To overcome such a limitation, this study presents an approach for predicting change-proneness based on the class change history. The approach adopts the sliding window method and is evaluated to obtain six kinds of models, which are derived by using, as predictors, different sets of metrics: structural, evolutionary, and smell-based. The evaluation uses five systems, four ML algorithms, and also explores some resample techniques to deal with imbalanced data. Regardless of the kind of model analyzed and the algorithm used, our approach overcomes the traditional one in 378 (~80) cases, out of 420, considering all systems, kinds of models, indicators, and algorithms. Moreover, the results show that our approach presents the best performance when the set of evolutionary metrics is used as predictors. There is no improvement when smell-based metrics are added. The Random Forest algorithm with the resampling technique ADA reaches the best performance among the ML algorithms evaluated.

Exploratory parallel hybrid sampling framework for imbalanced data classification

Current engineering application scenarios often face the challenge of imbalanced data, hybrid sampling is an effective method to deal with the imbalanced data classification issue, which can avoid the issues of overfitting and mistakenly deleting useful majority samples when using oversampling approach and undersampling approach alone. However, at present most of the hybrid sampling approaches are implemented serially, and the implementation of oversampling and undersampling approaches alone will cause mutual interference and influence between them. This study proposes a parallel hybrid sampling framework based on the idea of parallel engineering and theoretically analyzes its superiority. The experimental results show that when applied to five classification algorithms with three performance evaluation metrics,the proposed framework outperforms the two mainstream hybrid sampling frameworks. Moreover, the proposed framework can effectively reduce the time consumption of hybrid sampling process.

Explainable deep learning for glaucoma detection: Efficient dense net with quantum-based Dwarf Mongoose optimization

Glaucoma, a progressive eye disease, leads to irreversible vision loss due to optic nerve damage. This study aims to develop a more explainable and interpretable deep learning-based model for the detection and classification of glaucoma, addressing the current limitations in the field. The proposed approach utilizes a modified DenseNet-201 architecture, incorporating an additional transition layer for enhanced glaucoma classification. A pre-trained Efficient DenseNet is applied, and a reweighted cross-entropy loss function is used to handle the class imbalance in the training data. Quantum-based Dwarf Mongoose Optimization (QDMOA) is employed to fine-tune the model’s parameters, minimizing over fitting and improving generalization on small datasets. Experimental results demonstrate that the proposed method effectively detects and classifies two types of glaucoma. The use of dense connections with regularization significantly reduces over fitting, and the reweighted loss function improves model performance on imbalanced data. The developed deep learning model offers improved accuracy in glaucoma detection and classification while addressing the need for more interpretable and reliable models in medical applications. This research provides a practical tool for automated glaucoma screening, which could support ophthalmologists in early diagnosis and treatment, reducing the risk of irreversible vision loss.

Multiconstraint quality–probability graph for quality monitoring of laser directed energy deposition manufacturing process

Quality monitoring of the laser directed energy deposition (LDED) manufacturing process by machine vision is essential to improve the reliability and economy of the LDED manufactured parts. For monitoring algorithms, graph-based semi-supervised learning can effectively learn supervised information from labelled data without the requirement of large amounts of manually labelled data. However, traditional graph algorithm for LDED quality monitoring is limited by insufficient use of label supervised information, inadequate consideration of the imbalance data, and lack of physically meaningful explanations and constraints on the propagation process for different types of defects. In this regard, a multiconstraint quality–probability graph (MCQPG) is proposed to monitor the quality during the LDED manufacturing process. MCQPG converts the extracted multifeatures into probability distributions, finds feature sets with similar distributions to the supervised information based on quality level standards, and uses multiconstraints to satisfy few labelled feature data, imbalance data distribution and physical consistency. For physical consistency, a nonlinear defect model is developed for crack and porosity defects, and a novel defect-guided objective prediction function is proposed, resulting in the construction of a quality level standard guided physical consistency constraint term. Experimental studies on several well-known and commonly used monitoring algorithms demonstrate that the proposed MCQPG algorithm achieves 0.96 on all four evaluation metrics (accuracy, precision, recall and F1 score), validating the effectiveness of MCQPG for LDED quality monitoring.

Unravelling sleep patterns: Supervised contrastive learning with self-attention for sleep stage classification

Sleep data scoring is a crucial and primary step for diagnosing sleep disorders to know the sleep stages from the PSG signals. This study uses supervised contrastive learning with a self-attention mechanism to classify sleep stages. We propose a deep learning framework for automatic sleep stage classification, which involves two training phases: (1) the feature representation learning phase, in which the feature representation network (encoder) learns to extract features from the electroencephalogram (EEG) signals, and (2) the classification network training phase, where a pre-trained encoder (trained during phase I) along with the classifier head is fine-tuned for the classification task. The PSG data shows a non-uniform distribution of sleep stages, with wake (W) (around 30% of total samples) and N2 stages (around 58% and 37% of total samples in Physionet EDF-Sleep 2013 and 2018 datasets, respectively) being more prevalent, leading to an imbalanced dataset. The imbalanced data issue is addressed using a weighted softmax cross-entropy loss function that assigns higher weights to minority sleep stages. Additionally, an oversampling technique (the synthetic minority oversampling technique (SMOTE) (Chawla et al., 2002)[1] ) is applied to generate synthetic samples for minority classes. The proposed model is evaluated on the Physionet EDF-Sleep 2013 and 2018 datasets using Fpz-Cz and Pz-Oz EEG channels. It achieved an overall accuracy of 94.1%, a macro F1 score of 92.64, and a Cohen’s Kappa coefficient of 0.92. Ablation studies demonstrated the importance of triplet loss-based pre-training and oversampling for enhancing performance. The proposed model requires minimal pre-processing, eliminating the need for extensive signal processing expertise, and thus is well-suited for clinicians diagnosing sleep disorders.

A novel label-aware global graph construction method and spiking-coded graph neural network for intelligent process fault diagnosis

Fault diagnosis plays a crucial role in ensuring the safety and efficiency of industrial processes. However, traditional techniques often face difficulties in handling large-scale data characterized by complex structures and relationships. To efficiently represent industrial data as graphs and develop a low-energy-cost feature extraction model, a novel label-aware global graph construction method and a spiking graph convolutional network (SGCN) are proposed in this study to achieve intelligent process fault diagnosis. The label-aware method enhances graph data representation by capturing intrinsic correlations and global features. The SGCN integrates graph convolutional layers with spiking encoding, enabling effective feature extraction while offering computational efficiency advantages. A weighted loss function is introduced to mitigate data imbalance issues. Experiments on the Tennessee Eastman process, the Three-phase Flow Facility, and the de-propanizer distillation process demonstrate SGCN’s superior performance over baseline models in various fault scenarios, while significantly reducing computational costs. The proposed method offers promising potential for reliable and efficient fault diagnosis in complex real-world industrial environments.