Oversampling Method Research Articles

Cervical Cancer Prediction Based on Imbalanced Data Using Machine Learning Algorithms with a Variety of Sampling Methods

Cervical cancer affects a large portion of the female population, making the prediction of this disease using Machine Learning (ML) of utmost importance. ML algorithms can be integrated into complex, intelligent, agent-based systems that can offer decision support to resident medical doctors or even experienced medical doctors. For instance, an experienced medical doctor may diagnose a case but need expert support that related to another medical specialty. Data imbalance is frequent in healthcare data and has a negative influence on predictions made using ML algorithms. Cancer data, in general, and cervical cancer data, in particular, are frequently imbalanced. For this study, we chose a messy, real-life cervical cancer dataset available in the Kaggle repository that includes large amounts of missing and noisy values. To identify the best imbalanced technique for this medical dataset, the performances of eleven important resampling methods are compared, combined with the following state-of-the-art ML models that are frequently applied in predictive healtchare research: K-Nearest Neighbors (KNN) (with k values of 2 and 3), binary Logistic Regression (bLR), and Random Forest (RF). The studied resampling methods include seven undersampling methods and four oversampling methods. For this dataset, the imbalance ratio was 12.73, with a 95% confidence interval ranging from 9.23% to 16.22%. The obtained results show that resampling methods help improve the classification ability of prediction models applied to cervical cancer data. The applied oversampling techniques for handling imbalanced data generally outperformed the undersampling methods. The average balanced accuracy for oversampling was 77.44%, compared to 62.28% for undersampling. When detecting the minority class, oversampling achieved an average score of 60.80%, while undersampling scored 41.36%. The logistic regression classifier had the greatest impact on balanced techniques, while random forest achieved promising performance, even before applying balancing techniques. Initially, KNN2 outperformed KNN3 across all metrics, including balanced accuracy, for which KNN2 achieved 53.57%, compared to 52.71% for KNN3. However, after applying oversampling techniques, KNN3 significantly improved its balanced accuracy to 73.78%, while that of KNN2 increased to 63.89%. Additionally, KNN3 outperformed KNN2 in minority class performance, scoring 55.72% compared to KNN2’s 33.93%.

Random Oversampling-Based Diabetes Classification via Machine Learning Algorithms

Diabetes mellitus is considered one of the main causes of death worldwide. If diabetes fails to be treated and diagnosed earlier, it can cause several other health problems, such as kidney disease, nerve disease, vision problems, and brain issues. Early detection of diabetes reduces healthcare costs and minimizes the chance of serious complications. In this work, we propose an e-diagnostic model for diabetes classification via a machine learning algorithm that can be executed on the Internet of Medical Things (IoMT). The study uses and analyses two benchmarking datasets, the PIMA Indian Diabetes Dataset (PIDD) and the Behavioral Risk Factor Surveillance System (BRFSS) diabetes dataset, to classify diabetes. The proposed model consists of the random oversampling method to balance the range of classes, the interquartile range technique-based outlier detection to eliminate outlier data, and the Boruta algorithm for selecting the optimal features from the datasets. The proposed approach considers ML algorithms such as random forest, gradient boosting models, light gradient boosting classifiers, and decision trees, as they are widely used classification algorithms for diabetes prediction. We evaluated all four ML algorithms via performance indicators such as accuracy, F1 score, recall, precision, and AUC-ROC. Comparative analysis of this model suggests that the random forest algorithm outperforms all the remaining classifiers, with the greatest accuracy of 92% on the BRFSS diabetes dataset and 94% accuracy on the PIDD dataset, which is greater than the 3% accuracy reported in existing research. This research is helpful for assisting diabetologists in developing accurate treatment regimens for patients who are diabetic.

SGO: An innovative oversampling approach for imbalanced datasets using SVM and genetic algorithms

Imbalanced datasets present a challenging problem in machine learning and artificial intelligence. Since most models typically assume balanced data distributions, imbalanced positive and negative examples can lead to significant bias in prediction or classification tasks. Current over-sampling methods frequently encounter issues like overfitting and boundary bias. A novel imbalanced data augmentation technique called SVM-GA over-sampling (SGO) is proposed in this paper, which integrates Support Vector Machines (SVM) with Genetic Algorithms (GA). Our approach leverages SVM to identify the decision boundary and uses GA to generate new minority samples along this boundary, effectively addressing both over-fitting and boundary biases. It has been experimentally validated that SGO outperforms the traditional methods on most datasets, providing a novel and effective approach to address imbalanced data problems, with potential application prospects and generalization value.

Neutrosophic set and optimized deep learning for classification of chicken Eimeria species: a practical solution for poultry industry

AbstractTo optimize infection control and bolster productivity within the poultry industry, it is imperative to accurately classify Chicken Eimeria species. There are several methods for determining Eimeria disease in chickens. Traditional methods involve watching for clinical symptoms, and macroscopic lesions, and studying the parasite’s biology and oocyst morphology. These methods are frequently time-consuming and labor-intensive, necessitating the manual collection and analysis of samples, which can be especially difficult in large chicken farms. Deep learning algorithms, on the other hand, provide automated, accurate, and non-invasive methods for the detection of Eimeria. This paper proposed a classification model for the automatic classification of chicken Eimeria species. The proposed model is mainly based on integrating neutrosophic set theory and InceptionV3 deep-learning architecture. Three primary phases make up the proposed chicken Eimeria species classification model: the data preprocessing phase, the neutrosophic image conversion phase, and the image classification phase. To address the issue of class imbalance in the adopted dataset and enhance the model’s generalizability, the random oversampling method, and data augmentation techniques are employed during the data preprocessing phase. The preprocessed data is considered to feed the neutrosophic set-based segmentation algorithm, where true, false, and intermediate subsets are extracted. Finally, the true subset is utilized to feed the optimized InceptionV3. To determine the optimal hyperparameter values for InceptionV3, a modified version of the Brown Bear optimization algorithm is proposed in this paper. To evaluate the effectiveness of the proposed model, a real benchmark dataset comprising images of different Eimeria species is adopted. The experimental results revealed that the proposed model offers a more efficient and accurate alternative to traditional methods and state-of-the-art models, enabling faster and more effective diagnosis and treatment of Eimeria infections. It achieved an overall accuracy, specificity, sensitivity, and F1-score of nearly 100%. Additionally, the results showed that the high performance of the proposed model can reduce labor costs and boost throughput, thereby enhancing economic viability even more.

Generative AI with WGAN-GP for boosting seizure detection accuracy.

Imbalanced datasets pose challenges for developing accurate seizure detection systems based on electroencephalogram (EEG) data. Generative AI techniques may help augment minority class data to facilitate automatic epileptic seizure detection. This study investigates the impact of various data augmentation (DA) approaches, including Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP), Vanilla GAN, Conditional GAN (CGAN), and Cramer GAN, on classification performance with Random Forest models. The best-performing GAN variant, WGAN-GP, was then integrated with a bidirectional Long Short-Term Memory (LSTM) architecture and compared against traditional and synthetic oversampling methods. The evaluation of different GAN variants for data augmentation with Random Forest classifiers identified WGAN-GP as the most effective approach. The integration of WGAN-GP with bidirectional LSTM yielded substantial performance improvements, outperforming traditional oversampling methods and achieving an accuracy of 91.73% on the augmented data, compared to 86% accuracy on real data without augmentation. The proposed generative AI approach combining WGAN-GP and recurrent neural network models outperforms comparative synthetic oversampling methods on metrics relevant for reliable seizure detection from imbalanced EEG datasets. Incorporating the WGAN-GP generative AI technique for data augmentation and integrating it with bidirectional LSTM elevates seizure detection accuracy for imbalanced EEG datasets, surpassing the performance of traditional oversampling and class weight adjustment methods. This approach shows promise for improving epilepsy monitoring and management through enhanced automated detection system effectiveness.

HIDIM: A novel framework of network intrusion detection for hierarchical dependency and class imbalance

Deep learning-based network intrusion detection has been extensively explored as a data-driven approach. Therefore, paying attention to the data’s characteristics is essential. By analyzing the attribute dependence and sample distribution of intrusion data, there are the following problems: “hierarchical dependency omission” and “decision boundary discontinuity.” The former means the previous attribute embedding models failed to incorporate network protocol hierarchy. The latter indicates that the small disjuncts distribution leads to sub-concept fragmentation, exacerbating the difficulty in handling class imbalance. To address these problems, we propose a novel detection framework for Hierarchical Dependency and Class Imbalance (HIDIM). First, we treat semantic attributes as words and introduce the protocol hierarchy of attributes into a paragraph embedding model. Second, we design a synthetic oversampling method. It adopts a mutual nearest neighbour approach to determine the boundaries of each disjunct. Then, it synthesizes high-quality samples within those boundary areas by crossing or mutating features based on their importance. The experimental results on multiple real-world datasets demonstrate that the proposed framework is superior to other state-of-the-art models in terms of accuracy, F1-score, and false negative rate by 2.23%, 2.12%, and 1.43% on average, respectively.

Machine Learning for Enhanced Churn Prediction in Banking: Leveraging Oversampling and Stacking Techniques

Every sector of business is getting more competitive as time passes. More and more companies are offering services to people. Banking sector is no different. With the plethora option customer has in terms banking, holding on to customer may prove difficult for banks. This research will help banks to predict which customers are likely going to churn and allow them to take precaution to stop customers from leaving. In this study we have used classifiers such as: K-Neighbors, Random Forest, XGboost, Adaboost classifiers and Ensemble Model (Stacking Technique) that uses all of these models together. The experimentation was conducted on a dataset from Kaggle. The dataset used in this research was heavily imbalanced. So, different oversampling methods like Random Oversampling and SMOTE-ENN have been used. In data preprocessing, label encoding was done and for validation K-folding technique (k-5) have been used. The highest accuracy has been achieved by using Random Oversampling with Stacking Model which is 97.31% (std: 0.0033, k=5).

CTGAN-ENN: a tabular GAN-based hybrid sampling method for imbalanced and overlapped data in customer churn prediction

Class imbalance is one of many problems of customer churn datasets. One of the common problems is class overlap, where the data have a similar instance between classes. The prediction task of customer churn becomes more challenging when there is class overlap in the data training. In this research, we suggested a hybrid method based on tabular GANs, called CTGAN-ENN, to address class overlap and imbalanced data in datasets of customers that churn. We used five different customer churn datasets from an open platform. CTGAN is a tabular GAN-based oversampling to address class imbalance but has a class overlap problem. We combined CTGAN with the ENN under-sampling technique to overcome the class overlap. CTGAN-ENN reduced the number of class overlaps by each feature in all datasets. We investigated how effective CTGAN-ENN is in each machine learning technique. Based on our experiments, CTGAN-ENN achieved satisfactory results in KNN, GBM, XGB and LGB machine learning performance for customer churn predictions. We compared CTGAN-ENN with common over-sampling and hybrid sampling methods, and CTGAN-ENN achieved outperform results compared with other sampling methods and algorithm-level methods with cost-sensitive learning in several machine learning algorithms. We provide a time consumption algorithm between CTGAN and CTGAN-ENN. CTGAN-ENN achieved less time consumption than CTGAN. Our research work provides a new framework to handle customer churn prediction problems with several types of imbalanced datasets and can be useful in real-world data from customer churn prediction.

Oversampling multi-label data based on natural neighbor and label correlation

In multi-label learning, addressing the class imbalance issue is of paramount importance. Oversampling methods are preferred as they offer a more general solution independent of the model choice, i.e., they alleviate the imbalance of datasets by augmenting instances in the pre-processing step. Existing neighbor-based oversampling methods employ an empirical number of neighbors (k=5) to identify the local region for new instances creation. However, a single fixed k value cannot fit all labels, because every label usually has its own distinct distribution and complexity. Furthermore, the label assignment for synthetic instances usually depends on the statistics of individual labels within the corresponding neighborhood, ignoring the informative correlation among labels. To overcome these limitations, we propose an oversampling method called Multi-Label Oversampling with Natural neighbor and label Correlation (MLONC). Our approach offers three main advantages: (1) the adaptive number of neighbors for each label related to the data complexity is obtained via natural neighbor detection; (2) it encourages generating more instances proximate to the decision boundary of highly imbalanced labels, and diminishes the impact of outliers; (3) exploitation of label correlation in label assignment enhances the quality of the synthetic instances. Experimental results demonstrate the effectiveness of MLONC under various base classifiers.

An adaptive over-sampling method for imbalanced data based on simultaneous clustering and filtering noisy

An adaptive over-sampling method for imbalanced data based on simultaneous clustering and filtering noisy

A Multiwavelength Machine-learning Approach to Classifying X-Ray Sources in the Fields of Unidentified 4FGL-DR4 Sources

A large fraction of Fermi-Large Area Telescope (LAT) sources in the fourth Fermi-LAT 14 yr catalog (4FGL) still remain unidentified (unIDed). We continued to improve our machine-learning pipeline and used it to classify 1206 X-ray sources with signal-to-noise ratios >3 located within the extent of 73 unIDed 4FGL sources with Chandra X-ray Observatory observations included in the Chandra Source Catalog 2.0. Recent improvements to our pipeline include astrometric corrections, probabilistic cross-matching to lower-frequency counterparts, and a more realistic oversampling method. X-ray sources are classified into eight broad predetermined astrophysical classes defined in the updated training data set, which we also release. We present details of the machine-learning classification, describe the pipeline improvements, and perform an additional spectral and variability analysis for brighter sources. The classifications give 103 plausible X-ray counterparts to 42 GeV sources. We identify 2 GeV sources as isolated neutron star candidates, 16 as active galactic nucleus candidates, seven as sources associated with star-forming regions, and eight as ambiguous cases. For the remaining 40 unIDed 4FGL sources, we could not identify any plausible counterpart in X-rays, or they are too close to the Galactic Center. Finally, we outline the observational strategies and further improvements in the pipeline that can lead to more accurate classifications.

A Feature Engineering‐Based NILM Framework for Appliance Recognition Considering Data Class Imbalance

Due to the diversity and randomness of residential consumption behavior, as well as the different share of domestic electricity demand by appliances, class imbalance problems exist in the non‐intrusive load monitoring (NILM) system. The recognition model becomes biased toward the majority class, which makes the recognition of the minority class difficult. To tackle this challenge, we propose a novel NILM framework for appliance recognition to overcome the insufficient learning problems of the minority class, which combines a multi‐domain feature extraction module, a two‐stage feature selection, and a data oversampling method. Multiple features are extracted to make full of the complementarity of combined features to enhance appliance recognition. Furthermore, a two‐stage feature selection based on minimal‐redundancy‐maximal‐relevance (mRMR) and random forest (RF) is developed to select an optimal feature set. Subsequently, an oversampling method combining K‐medoids and the synthetic minority oversampling technique (KM‐SMOTE) with sampling weights is proposed for data augmentation of the minority class to handle imbalance learning problems. The effectiveness of the proposed method is verified by the experiments on the public dataset PLAID. The results show that, compared with the state‐of‐the‐art techniques, the overall recognition accuracy and F1‐score with the proposed method are enhanced by 4.61% and 2.04%, respectively. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

An Oversampling Technique with Descriptive Statistics

Oversampling is often applied as a means to win a better knowledge model. Several oversampling methods based on synthetic instances have been suggested, and SMOTE is one of the representative oversampling methods that can generate synthetic instances of a minor class. Until now, the oversampled data has been used conventionally to train machine learning models without statistical analysis, so it is not certain that the machine learning models will be fine for unseen cases in the future. However, because such synthetic data is different from the original data, we may wonder how much it resembles the original data so that the oversampled data is worth using to train machine learning models. For this purpose, I conducted this study on a representative dataset called wine data in the UCI machine learning repository, which is one of the datasets that has been experimented with by many researchers in research for knowledge discovery models. I generated synthetic data iteratively using SMOTE, and I compared the synthetic data with the original data of wine to see if it was statistically reliable using a box plot and t-test. Moreover, since training a machine learning model by supplying more high-quality training instances increases the probability of obtaining a machine learning model with higher accuracy, it was also checked whether a better machine learning model of random forests can be obtained by generating much more synthetic data than the original data and using it for training the random forests. The results of the experiment showed that small-scale oversampling produced synthetic data with statistical characteristics that were statistically slightly different from the original data, but when the oversampling rate was relatively high, it was possible to generate data with statistical characteristics similar to the original data, in other words, after generating high-quality training data, and by using it to train the random forests, it was possible to generate random forests with higher accuracy than using the original data alone, from 97.75% to 100%. Therefore, by supplying additional statistically reliable synthetic data as a way of oversampling, it was possible to create a machine-learning model with a higher predictive rate.

ANN-assisted prediction of wave run-up around a tension leg platform under irregular wave conditions

This study proposes a novel prediction method for nonlinear wave run-ups around a TLP by combining the linear diffraction method with deep learning techniques. Initially, the linear diffraction method evaluates relative wave motion around the platform based on measured incident waves and frequency domain analysis. Subsequently, an artificial neural network (ANN) predicts the peak values of nonlinear wave run-ups, utilizing incident wave data and linear calculation results as inputs. Two different ANN models were proposed, each tailored to specific output variables. The first, the ζ-based ANN model, predicts the peak value of wave run-ups, while the second, the α-based ANN model, predicts the nonlinear amplification factor. The incorporation of a neural network significantly enhances the prediction accuracy for the peak values of wave run-ups. Moreover, the α-based ANN model demonstrates superior accuracy in predicting high run-up events and better generalization ability across various wave conditions compared to the ζ-based ANN model. The study also investigates the effect of various preprocessing methods on prediction performance. The over-sampling method improves accuracy for high run-up events but degrades predictions for low run-up regions, while the polynomial feature transformation reduces prediction errors for high run-up in the ζ-based ANN model.

Oversampling for Enhanced Spatial Resolution of Zebrafish by Top-Hat IR-MALDESI-MSI.

Mass spectrometry imaging (MSI) has become a significant tool for measuring chemical species in biological tissues, where much of the impact of these platforms lies in their capability to report the spatial distribution of analytes for correlation to sample morphology. As a result, enhancement of spatial resolution has become a frontier of innovation in the field, and necessary developments are dependent on the ionization source. More particularly, laser-based imaging sources may require modifications to the optical train or alternative sampling techniques. These challenges are heightened for systems with infrared (IR) lasers, as their operating wavelength generates spot sizes that are inherently larger than their ultraviolet counterparts. Recently, the infrared matrix-assisted laser desorption electrospray ionization (IR-MALDESI) source has shown the utility of a diffractive optical element (DOE) to produce square ablation patterns, termed top-hat IR-MALDESI. If the DOE optic is combined with oversampling methods, smaller ablation volumes can be sampled to render higher spatial resolution imaging experiments. Further, this approach enables reproducible spot sizes and ablation volumes for better comparison between scans. Herein, we investigate the utility of oversampling with top-hat IR-MALDESI to enhance the spatial resolution of measured lipids localized within the head of sectioned zebrafish tissue. Four different spatial resolutions were evaluated for data quality (e.g., mass measurement accuracy, spectral accuracy) and quantity of annotations. Other experimental parameters to consider for high spatial resolution imaging are also discussed. Ultimately, 20 μm spatial resolution was achieved in this work and supports feasibility for use in future IR-MALDESI studies.

Visual-based vehicle detection with adaptive oversampling

In this study, the class imbalance issue in vehicle detection was addressed. Specifically, certain classes such as Tow Truck were found to have significantly fewer samples compared to others such as normal trucks. This imbalance could be adversely impacted algorithm performance, favouring abundant classes over underrepresented ones. After thorough analysis, an adaptive dataset augmentation approach was proposed for the underrepresented classes. Evaluation was first performed on classic and state-of-the-art object detection methods. All experiments were undertaken on a tiny dataset called Multimedia University Diversity Dataset (MMUVD). The fastest training process and the highest mean average precision (mAP), which stood at 0.686 for mAP50 and 0.439 for mAP50-95, were demonstrated by You Only Look Once version 8 nano (YOLOv8n). By applying adaptive oversampling to the dataset and retesting it again on YOLOv8n, mAP50 was improved to 0.950 and mAP50-95 to 0.717, respectively. Notably, the contribution lay in identifying the optimal detection algorithm for vehicle detection, and the proposed adaptive oversampling method ensured consistent performance across all classes, enhancing the overall accuracy and reliability of the system.

Enhancing customer retention in telecom industry with machine learning driven churn prediction

Customer churn remains a critical concern for businesses, highlighting the significance of retaining existing customers over acquiring new ones. Effective prediction of potential churners aids in devising robust retention policies and efficient customer management strategies. This study dives into the realm of machine learning algorithms for predictive analysis in churn prediction, addressing the inherent challenge posed by diverse and imbalanced customer churn data distributions. This paper introduces a novel approach—the Ratio-based data balancing technique, which addresses data skewness as a pre-processing step, ensuring improved accuracy in predictive modelling. This study fills gaps in existing literature by highlighting the effectiveness of ensemble algorithms and the critical role of data balancing techniques in optimizing churn prediction models. While our research contributes a novel approach, there remain avenues for further exploration. This work evaluates several machine learning algorithms—Perceptron, Multi-Layer Perceptron, Naive Bayes, Logistic Regression, K-Nearest Neighbour, Decision Tree, alongside Ensemble techniques such as Gradient Boosting and Extreme Gradient Boosting (XGBoost)—on balanced datasets achieved through our proposed Ratio-based data balancing technique and the commonly used Data Resampling. Results reveal that our proposed Ratio-based data balancing technique notably outperforms traditional Over-Sampling and Under-Sampling methods in churn prediction accuracy. Additionally, using combined algorithms like Gradient Boosting and XGBoost showed better results than using single methods. Our study looked at different aspects like Accuracy, Precision, Recall, and F-Score, finding that these combined methods are better for predicting customer churn. Specifically, when we used a 75:25 ratio with the XGBoost method, we got the most promising results for our analysis which are presented in this work.

An interpretability model for syndrome differentiation of HBV-ACLF in traditional Chinese medicine using small-sample imbalanced data

ObjectiveClinical medical record data associated with hepatitis B-related acute-on-chronic liver failure (HBV-ACLF) generally have small sample sizes and a class imbalance. However, most machine learning models are designed based on balanced data and lack interpretability. This study aimed to propose a traditional Chinese medicine (TCM) diagnostic model for HBV-ACLF based on the TCM syndrome differentiation and treatment theory, which is clinically interpretable and highly accurate. MethodsWe collected medical records from 261 patients diagnosed with HBV-ACLF, including three syndromes: Yang jaundice (214 cases), Yang-Yin jaundice (41 cases), and Yin jaundice (6 cases). To avoid overfitting of the machine learning model, we excluded the cases of Yin jaundice. After data standardization and cleaning, we obtained 255 relevant medical records of Yang jaundice and Yang-Yin jaundice. To address the class imbalance issue, we employed the oversampling method and five machine learning methods, including logistic regression (LR), support vector machine (SVM), decision tree (DT), random forest (RF), and extreme gradient boosting (XGBoost) to construct the syndrome diagnosis models. This study used precision, F1 score, the area under the receiver operating characteristic (ROC) curve (AUC), and accuracy as model evaluation metrics. The model with the best classification performance was selected to extract the diagnostic rule, and its clinical significance was thoroughly analyzed. Furthermore, we proposed a novel multiple-round stable rule extraction (MRSRE) method to obtain a stable rule set of features that can exhibit the model’s clinical interpretability. ResultsThe precision of the five machine learning models built using oversampled balanced data exceeded 0.90. Among these models, the accuracy of RF classification of syndrome types was 0.92, and the mean F1 scores of the two categories of Yang jaundice and Yang-Yin jaundice were 0.93 and 0.94, respectively. Additionally, the AUC was 0.98. The extraction rules of the RF syndrome differentiation model based on the MRSRE method revealed that the common features of Yang jaundice and Yang-Yin jaundice were wiry pulse, yellowing of the urine, skin, and eyes, normal tongue body, healthy sublingual vessel, nausea, oil loathing, and poor appetite. The main features of Yang jaundice were a red tongue body and thickened sublingual vessels, whereas those of Yang-Yin jaundice were a dark tongue body, pale white tongue body, white tongue coating, lack of strength, slippery pulse, light red tongue body, slimy tongue coating, and abdominal distension. This is aligned with the classifications made by TCM experts based on TCM syndrome differentiation and treatment theory. ConclusionOur model can be utilized for differentiating HBV-ACLF syndromes, which has the potential to be applied to generate other clinically interpretable models with high accuracy on clinical data characterized by small sample sizes and a class imbalance.

Imbalanced Data Classification Based on Improved Random-SMOTE and Feature Standard Deviation

Oversampling techniques are widely used to rebalance imbalanced datasets. However, most of the oversampling methods may introduce noise and fuzzy boundaries for dataset classification, leading to the overfitting phenomenon. To solve this problem, we propose a new method (FSDR-SMOTE) based on Random-SMOTE and Feature Standard Deviation for rebalancing imbalanced datasets. The method first removes noisy samples based on the Tukey criterion and then calculates the feature standard deviation reflecting the degree of data discretization to detect the sample location, and classifies the samples into boundary samples and safety samples. Secondly, the K-means clustering algorithm is employed to partition the minority class samples into several sub-clusters. Within each sub-cluster, new samples are generated based on random samples, boundary samples, and the corresponding sub-cluster center. The experimental results show that the average evaluation value obtained by FSDR-SMOTE is 93.31% (93.16%, and 86.53%) in terms of the F-measure (G-mean, and MCC) on the 20 benchmark datasets selected from the UCI machine learning library.

Applying oversampling before cross-validation will lead to high bias in radiomics

Class imbalance is often unavoidable for radiomic data collected from clinical routine. It can create problems during classifier training since the majority class could dominate the minority class. Consequently, resampling methods like oversampling or undersampling are applied to the data to class-balance the data. However, the resampling must not be applied upfront to all data because it would lead to data leakage and, therefore, to erroneous results. This study aims to measure the extent of this bias. Five-fold cross-validation with 30 repeats was performed using a set of 15 radiomic datasets to train predictive models. The training involved two scenarios: first, the models were trained correctly by applying the resampling methods during the cross-validation. Second, the models were trained incorrectly by performing the resampling on all the data before cross-validation. The bias was defined empirically as the difference between the best-performing models in both scenarios in terms of area under the receiver operating characteristic curve (AUC), sensitivity, specificity, balanced accuracy, and the Brier score. In addition, a simulation study was performed on a randomly generated dataset for verification. The results demonstrated that incorrectly applying the oversampling methods to all data resulted in a large positive bias (up to 0.34 in AUC, 0.33 in sensitivity, 0.31 in specificity, and 0.37 in balanced accuracy). The bias depended on the data balance, and approximately an increase of 0.10 in the AUC was observed for each increase in imbalance. The models also showed a bias in calibration measured using the Brier score, which differed by up to −0.18 between the correctly and incorrectly trained models. The undersampling methods were not affected significantly by bias. These results emphasize that any resampling method should be applied correctly only to the training data to avoid data leakage and, subsequently, biased model performance and calibration.