Imbalanced Datasets Research Articles

Analysis of ensemble machine learning classification comparison on the skin cancer MNIST dataset

This study aims to analyze the performance of various ensemble machine learning methods, such as Adaboost, Bagging, and Stacking, in the context of skin cancer classification using the skin cancer MNIST dataset. We also evaluate the impact of handling dataset imbalance on the classification model’s performance by applying imbalanced data methods such as random under sampling (RUS), random over sampling (ROS), synthetic minority over-sampling technique (SMOTE), and synthetic minority over-sampling technique with edited nearest neighbor (SMOTEENN). The research findings indicate that Adaboost is effective in addressing data imbalance, while imbalanced data methods can significantly improve accuracy. However, the selection of imbalanced data methods should be carefully tailored to the dataset characteristics and clinical objectives. In conclusion, addressing data imbalance can enhance skin cancer classification accuracy, with Adaboost being an exception that shows a decrease in accuracy after applying imbalanced data methods.

Integrating Autoencoders with Local Outlier Factor and Isolation Forest for Effective Fraud Detection in Imbalanced Datasets

Integrating Autoencoders with Local Outlier Factor and Isolation Forest for Effective Fraud Detection in Imbalanced Datasets

A Comprehensive Survey on Brain Tumor Detection and Classification Techniques Using Machine Learning and Deep Learning Models

Abstract: Brain tumor detection and classification have become critical areas of research due to the complexities and challenges involved in accurate diagnosis. With the advancement of machine learning and deep learning techniques, numerous intelligent systems have been proposed to automate and enhance the process of detecting and classifying brain tumors. This survey paper reviews recent approaches and techniques used for brain tumor detection, focusing primarily on deep learning architectures like Convolutional Neural Networks (CNNs) and pre-trained models such as VGG, ResNet, and EfficientNet. In particular, methods using MRI images to classify tumors into categories such as glioma, meningioma, pituitary tumors, and healthy tissue are explored. A detailed analysis of various models, their architectures, training strategies, and evaluation metrics is provided. Moreover, the paper identifies the key challenges faced in the field, including imbalanced datasets, generalization issues, and computational efficiency. By comparing different approaches, we aim to highlight the strengths and limitations of each method and provide insights into how future research can address these challenges. The paper concludes with a discussion on promising future directions in the application of intelligent techniques for brain tumor detection, including the integration of multimodal data and the development of more interpretable models. This survey serves as a comprehensive resource for researchers interested in understanding the current landscape of brain tumor detection using advanced deep learning techniques.

AugmenToxic: Leveraging Reinforcement Learning to Optimize LLM Instruction Fine-Tuning for Data Augmentation to Enhance Toxicity Detection

Addressing the challenge of toxic language in online discussions is crucial for the development of effective toxicity detection models. This pioneering work focuses on addressing imbalanced datasets in toxicity detection by introducing a novel approach to augment toxic language data. We create a balanced dataset by instructing fine-tuning of Large Language Models (LLMs) using Reinforcement Learning with Human Feedback (RLHF). Recognizing the challenges in collecting sufficient toxic samples from social media platforms for building a balanced dataset, our methodology involves sentence-level text data augmentation through paraphrasing existing samples using optimized generative LLMs. Leveraging generative LLM, we utilize the Proximal Policy Optimizer (PPO) as the RL algorithm to fine-tune the model further and align it with human feedback. In other words, we start by fine-tuning a LLM using an instruction dataset, specifically tailored for the task of paraphrasing while maintaining semantic consistency. Next, we apply PPO and a reward function, to further fine-tune (optimize) the instruction-tuned LLM. This RL process guides the model in generating toxic responses. We utilize the Google Perspective API as a toxicity evaluator to assess generated responses and assign rewards/penalties accordingly. This approach guides LLMs through PPO and the reward function, transforming minority class samples into augmented versions. The primary goal of our methodology is to create a balanced and diverse dataset to enhance the accuracy and performance of classifiers in identifying instances from the minority class. Utilizing two publicly available toxic datasets, we compared various techniques with our proposed method for generating toxic samples, demonstrating that our approach outperforms all others in producing a higher number of toxic samples. Starting with an initial 16,225 toxic prompts, our method successfully generated 122,951 toxic samples with a toxicity score exceeding 30%. Subsequently, we developed various classifiers using the generated balanced datasets and applied a cost-sensitive learning approach to the original imbalanced dataset. The findings highlight the superior performance of classifiers trained on data generated using our proposed method. These results highlight the importance of employing RL and a data-agnostic model as a reward mechanism for augmenting toxic data, thereby enhancing the robustness of toxicity detection models.

Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores

This study compares various F1-score variants—micro, macro, and weighted—to assess their performance in evaluating text-based emotion classification. Lexicon distillation is employed using the multilabel emotion-annotated datasets XED and GoEmotions. The aim of this paper is to understand when each F1-score variant is better suited for evaluating text-based multilabel emotion classification. Unigram lexicons were derived from the annotated GoEmotions and XED datasets through a binary classification approach. The distilled lexicons were then applied to the GoEmotions and XED annotated datasets to calculate their emotional content, and the results were compared. The findings highlight the behavior of each F1-score variant under different class distributions, emphasizing the importance of appropriate metric selection for reliable model performance evaluation in imbalanced multilabel datasets. Additionally, this study also investigates the effect of the aggregation of negative emotions into broader categories on said F1 metrics. The contribution of this study is to provide insights into how different F1-score variants could improve the reliability of multilabel emotion classifier evaluation, particularly in the context of class imbalance present in the case of phishing emails.

Improvement The Accuracy of Convolutional Neural Network with Using Undersampling Method on Unbalanced Credit Card Dataset

In this study, we address the challenge of imbalanced data in credit card fraud detection by proposing a novel approach that leverages Convolutional Neural Networks (CNNs) and undersampling techniques. The imbalance in the dataset, typical of real-world financial transactions, often leads to biased models favoring the majority class. To mitigate this, we employ undersampling to balance the classes, thereby enhancing the CNN's ability to learn from minority instances crucial for fraud detection. Our method is validated on a large unbalanced credit card dataset, demonstrating significant improvements in accuracy compared to traditional CNN models trained on imbalanced data. We evaluate our approach using standard performance metrics, including precision, recall, and F1-score, showcasing its effectiveness in accurately identifying fraudulent transactions while minimizing false positives. Furthermore, we pro-vide insights into the CNN's decision-making process through visualization techniques, shedding light on its ability to discern fraudulent patterns within the data. Our findings highlight the importance of addressing class imbalance in fraud detection tasks and underscore the efficacy of undersampling in enhancing the performance of deep learning models, particularly CNNs, in handling imbalanced datasets.

Enhancing Machine Learning Models Through PCA, SMOTE-ENN, and Stochastic Weighted Averaging

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

A Comprehensive Analysis of a Framework for Rebalancing Imbalanced Medical Data Using an Ensemble-based Classifier

In medical data, addressing imbalanced datasets is paramount for accurate predictive modeling. This paper delves into exploring a well-established rebalancing framework proposed in previous research. While acknowledged for its effectiveness, the adaptability of this framework across diverse medical datasets remains unexplored. We conduct a comprehensive investigation to bridge this gap by integrating an ensemble-based classifier into the existing framework. By leveraging seven imbalanced medical binary datasets, our study comprises three distinct experiments: utilizing standard baseline classifiers from the framework (original), incorporating the baseline with an ensemble-based classifier, and introducing our novel ensemble-based classifier with the self-paced ensemble (SPE) algorithm. Our novel ensemble, composed of decision tree (DT), radial support vector machine (R.SVM), and extreme gradient boosting (XGB) classifiers, serves as the foundation for the SPE. Our primary objective is to demonstrate the potential improvement of the existing framework’s overall performance through the integration of an ensemble. Experimental results reveal significant enhancements, with our proposed ensemble classifier outperforming the original by 4.96%, 5.89%, 5.68%, 7.85%, and 6.84% in terms of accuracy, precision, recall, F-score, and G-mean, respectively. This study contributes valuable insights into the adaptability and performance augmentation achievable through ensemble methods in addressing class imbalances within the medical domain.

Impacts of Data Preprocessing and Sampling Techniques on Solar Flare Prediction from Multivariate Time Series Data of Photospheric Magnetic Field Parameters

The accurate prediction of solar flares is crucial due to their risks to astronauts, space equipment, and satellite communication systems. Our research enhances solar flare prediction by employing sophisticated data preprocessing and sampling techniques for the Space Weather Analytics for Solar Flares (SWAN-SF) data set, a rich source of multivariate time series data of solar active regions. Our study adopts a multifaceted approach encompassing four key methodologies. Initially, we address over 10 million missing values in the SWAN-SF data set through our innovative imputation technique called fast Pearson correlation-based k-nearest neighbors imputation. Subsequently, we propose a precise normalization technique, called LSBZM normalization, tailored for time series data, merging various strategies (log, square root, Box–Cox, Z-score, and min–max) to uniformly scale the data set's 24 attributes (photospheric magnetic field parameters), addressing issues such as skewness. We also explore the “near decision boundary sample removal” technique to enhance the classification performance of the data set by effectively resolving the challenge of class overlap. Finally, a pivotal aspect of our research is a thorough evaluation of diverse oversampling and undersampling methods, including SMOTE, ADASYN, Gaussian noise injection, TimeGAN, Tomek links, and random undersampling, to counter the severe imbalance in the SWAN-SF data set, notably a 60:1 ratio of major (X and M) to minor (C, B, and FQ) flaring events in binary classification. To demonstrate the effectiveness of our methods, we use eight classification algorithms, including advanced deep-learning-based architectures. Our analysis shows significant true skill statistic scores, underscoring the importance of data preprocessing and sampling in time-series-based solar flare prediction.

OUCH: Oversampling and Undersampling Cannot Help Improve Accuracy in Our Bayesian Classifiers That Predict Preeclampsia

Unbalanced data can have an impact on the machine learning (ML) algorithms that build predictive models. This manuscript studies the influence of oversampling and undersampling strategies on the learning of the Bayesian classification models that predict the risk of suffering preeclampsia. Given the properties of our dataset, only the oversampling and undersampling methods that operate with numerical and categorical attributes will be taken into consideration. In particular, synthetic minority oversampling techniques for nominal and continuous data (SMOTE-NC), SMOTE—Encoded Nominal and Continuous (SMOTE-ENC), random oversampling examples (ROSE), random undersampling examples (UNDER), and random oversampling techniques (OVER) are considered. According to the results, when balancing the class in the training dataset, the accuracy percentages do not improve. However, in the test dataset, both positive and negative cases of preeclampsia were accurately classified by the models, which were built on a balanced training dataset. In contrast, models built on the imbalanced training dataset were not good at detecting positive cases of preeclampsia. We can conclude that while imbalanced training datasets can be addressed by using oversampling and undersampling techniques before building prediction models, an improvement in model accuracy is not always guaranteed. Despite this, the sensitivity and specificity percentages improve in binary classification problems in most cases, such as the one we are dealing with in this manuscript.

An Improved Design for a Cloud Intrusion Detection System Using Hybrid Features Selection Approach with ML Classifier

The rapid adoption of cloud computing has introduced new security concerns, particularly regarding data privacy and protection from sophisticated cyberattacks. Intrusion Detection Systems (IDSs) based on Machine Learning (ML) offer improved detection of malicious behaviour in network traffic. However, existing IDS solutions struggle with large-dimensional data and imbalanced datasets, resulting in higher false positive rates (FPR) and reduced detection rates (DR). In this paper, we propose an improved Cloud IDS leveraging a hybrid features selection approach using Information Gain (IG), Chi-square (CS), and Particle Swarm Optimization (PSO). To address the challenge of imbalanced datasets, the Synthetic Minority Over-sampling Technique (SMOTE) is employed. The Random Forest (RF) classifier is used to detect and classify attacks. The system is evaluated on the UNSW-NB15 and Kyoto datasets, achieving accuracies of over 98% and 99%, respectively, outperforming other methods in terms of detection accuracy.

SGO: An innovative oversampling approach for imbalanced datasets using SVM and genetic algorithms

Imbalanced datasets present a challenging problem in machine learning and artificial intelligence. Since most models typically assume balanced data distributions, imbalanced positive and negative examples can lead to significant bias in prediction or classification tasks. Current over-sampling methods frequently encounter issues like overfitting and boundary bias. A novel imbalanced data augmentation technique called SVM-GA over-sampling (SGO) is proposed in this paper, which integrates Support Vector Machines (SVM) with Genetic Algorithms (GA). Our approach leverages SVM to identify the decision boundary and uses GA to generate new minority samples along this boundary, effectively addressing both over-fitting and boundary biases. It has been experimentally validated that SGO outperforms the traditional methods on most datasets, providing a novel and effective approach to address imbalanced data problems, with potential application prospects and generalization value.

An advancement in AdaSyn for imbalanced learning: An application to fraud detection in digital transactions

Imbalanced Learning is a significant issue in machine learning, affecting the performance and accuracy of binary or multi-classification algorithms, especially in large-scale data handling and classification. There are some popular techniques to covert this imbalanced data into a balanced one such as undersampling, under-sampling with tomek links, randomized oversampling, synthetic minority oversampling technique (SMOTE), and adaptive synthetic generation (ADASYN). Generally, the ADASYN algorithm could be used to propagate minority sample points to rise the imbalanced ratio between majority and minority sample points, but in some cases, it may conflict with decision boundary points and noisy points. This paper proposed a Refitted AdaSyn Algorithm (RAA) with Gaussian Distribution (GD). So that new minority samples are distributed much closer to the center of the minority sample to spotlight the conflicts. The classification accuracy has improved with RAA over formal ADASYN. For examining the proposed work the imbalanced benchmark datasets like European, Banksim, Paymentcard, and UCI credit card are considered. Vanilla Generative Adversarial Network (GAN) is a deep learning model used to classify fraud and non-fraud transactions, demonstrating significant differences between balanced and imbalanced learning approaches and achieving an accuracy of 97.5% on dataset DS4.

A Supervised Ml Algorithm for Detecting and Predicting Fraud Credit Card Transactions

A Credit card fraud presents an escalating threat in today's digital economy, leading to significant financial losses for consumers, businesses, and financial institutions. Traditional detection methods are increasingly inadequate due to the rise in online transactions and the evolving complexity of fraudulent activities. This research aims to create a reliable supervised machine learning model designed to detect and predict fraudulent credit card transactions in real-time. Our approach leverages advanced algorithms and extensive datasets to enhance transaction security, thereby mitigating financial risks. Current systems, like FICO's Fraud Detection System (FDS), utilize a range of supervised learning techniques, including Decision Trees, Random Forests, and Neural Networks. However, they face limitations, including data quality issues, high false positive rates, and the need for continuous retraining to adapt to evolving fraud patterns. To address these challenges, we propose an innovative anomaly detection method based on the Isolation Forest algorithm. Unlike traditional methods that rely on distance and density measures, Isolation Forests isolate anomalies by randomly selecting features and split values, making it more efficient in identifying fraudulent transactions, especially in imbalanced datasets. The proposed system boasts low linear time complexity and minimal memory requirements, enabling the construction of high-performing models even with large datasets. The research demonstrates that the Isolation Forest approach significantly outperforms traditional models in detecting credit card fraud, offering a promising solution for enhancing security in the financial ecosystem.

Balancing data imbalance in biomedical datasets using a stacked augmentation approach with STDA, DAGAN, and pufferfish optimization to reveal AI's transformative impact

Balancing data imbalance in biomedical datasets using a stacked augmentation approach with STDA, DAGAN, and pufferfish optimization to reveal AI's transformative impact

Imbalanced graph learning via mixed entropy minimization

Imbalanced datasets, where the minority class is underrepresented, pose significant challenges for node classification in graph learning. Traditional methods often address this issue through synthetic oversampling techniques for the minority class, which can complicate the training process. To address these challenges, we introduce a novel training paradigm for node classification on imbalanced graphs, based on mixed entropy minimization (ME). Our proposed method, GraphME, offers a ‘free imbalance defense’ against class imbalance without requiring additional steps to improve classification performance. ME aims to achieve the same goal as cross-entropy-maximizing the model’s probability for the correct classes-while effectively reducing the impact of incorrect class probabilities through a “guidance” term that ensures a balanced trade-off. We validate the effectiveness of our approach through experiments on multiple datasets, where GraphME consistently outperforms the traditional cross-entropy objective, demonstrating enhanced robustness. Moreover, our method can be seamlessly integrated with various adversarial training techniques, leading to substantial improvements in robustness. Notably, GraphME enhances classification accuracy without compromising efficiency, a significant improvement over existing methods. The GraphME code is available at: https://github.com/12chen20/GraphME.

RSBagging: An ensemble classifier detecting the after-effects of ischemic stroke through EEG connectivity and microstates.

Stroke can lead to significant after-effects, including motor function impairments, language impairments (aphasia), disorders of consciousness (DoC), and cognitive deficits. Computer-aided analysis of EEG connectivity matrices and microstates from bedside EEG monitoring can replace traditional clinical observation methods, offering an automatic approach to monitoring the progression of these after-effects. This EEG-based method also enables quicker and more efficient assessments for medical practitioners. In this study, we employed Functional Connectivity features that extract spatial representation and Microstate features that focus on the time domain representation to monitor the after-effects of ischemic stroke patients. As the dataset from stroke patients is heavily imbalanced across various clinical after-effects conditions, we designed an ensemble classifier, RSBagging, to address the issue of classifiers often favoring the majority classes in the classification of imbalanced datasets. The experimental results demonstrate that different connectivity matrices are effective for three classification tasks: consciousness level, motor disturbance, and stroke location. Using our RSBagging model, all three tasks achieve over 98% accuracy, sensitivity, specificity, and F1-score, significantly outperforming the existing classifiers SVM, XGBoost, and Random Forest. Therefore, the RSBagging classifier based on connectivity matrices offers an effective method for monitoring the after-effects in stroke patients.

DRFL: Dynamic-Recall Focal Loss for Image Classification and Segmentation

ABSTRACT The accuracy of neural networks heavily depends on numerous precisely annotated samples. But in actual datasets, the number of samples for each category varies greatly. In the dataset, some classes may have a large sample size, called majority classes, while others may have a small sample size, called minority classes. Training model with imbalanced data is often conducive to the majority classes, while unfair to minority classes. As a result, the outputs return good performance on majority classes and bad performance on minority classes. This article proposes a new loss function called dynamic-recall focal loss (DRFL), which can solve the problem of imbalanced data categories in image classification and medical segmentation tasks. The DRFL assigns different weight coefficient to the classes according to their dynamic recall based on focal loss. Experimental results have shown that the newly proposed loss function DRFL can effectively improve the classification and segmentation accuracy of two imbalanced datasets.

PRO-SMOTEBoost: An adaptive SMOTEBoost probabilistic algorithm for rebalancing and improving imbalanced data classification

In the field of data mining and machine learning, dealing with imbalanced datasets is one of the most complex problems. The class imbalance issue significantly affects the classification of minority classes when using common classification algorithms. These algorithms often prioritize improving the performance of the majority class at the expense of the minority class, leading to misclassifying negative instances as positive ones. To address this problem, the Synthetic Minority Over-sampling Technique (SMOTE) has gained popularity to rebalance imbalanced data for classification. However, in this paper, we propose two algorithms to enhance the performance of imbalanced classification further. The first algorithm is PRO-SMOTE, an improvement over SMOTE. PRO-SMOTE relies on conditional probabilities to effectively rebalance imbalanced classes and improve the predictive performance metrics satisfactorily and reliably. By considering conditional probabilities, PRO-SMOTE can reduce the majority classes and optimally increase the minority class. Second, the PRO-SMOTEBoost algorithm, in turn, is based on the PRO-SMOTE to overcome classification anomalies and problems encountered by machine learning algorithms during classification, especially the weak ones. PRO-SMOTEBoost aims to maximize predictive precision to the greatest extent possible by combining the strengths of PRO-SMOTE with boosting techniques. Evaluating these algorithms using traditional machine learning algorithms such as Random Forests, C4.5, Naive Bayes, and Support Vector Machines has demonstrated excellent classification results. The performance metrics, encompassing F1-score, G-means, Precision, Accuracy, Recall, AUC-ROC, and Precision-Recall-curves, achieved by the proposed algorithm demonstrate a range that extends from over 90% to a flawless score of 100%. Compared to using these traditional algorithms individually, the utilization of PRO-SMOTEBoost has shown a significant improvement of 10% to 40% in performance metrics. Overall, the proposed algorithms, PRO-SMOTE and PRO-SMOTEBoost, offer effective solutions to address the challenges posed by imbalanced datasets. They provide improved predictive metrics and demonstrate their superiority when compared to traditional even modern machine learning algorithms.

Improving the Imbalanced Data Accuracy Using CNN and ReLU

In today’s academic world, learning from datasets is a trendy topic. In the event of imbalanced data, none of the many data mining tools available for this purpose work well, partly because this type of data generates a range of minority classes, which might obstruct the learning process. In addition to its enormous volume, big data has the traits of speed and variety. In this paper, the authors have proposed a CNN model with a rectified linear activation function (ReLU) activation function to get good accuracy on an imbalanced dataset. The dataset used in this paper was electrocardiogram (ECG) heartbeat categorization.