Undersampling Approach Research Articles

Exploratory parallel hybrid sampling framework for imbalanced data classification

Current engineering application scenarios often face the challenge of imbalanced data, hybrid sampling is an effective method to deal with the imbalanced data classification issue, which can avoid the issues of overfitting and mistakenly deleting useful majority samples when using oversampling approach and undersampling approach alone. However, at present most of the hybrid sampling approaches are implemented serially, and the implementation of oversampling and undersampling approaches alone will cause mutual interference and influence between them. This study proposes a parallel hybrid sampling framework based on the idea of parallel engineering and theoretically analyzes its superiority. The experimental results show that when applied to five classification algorithms with three performance evaluation metrics,the proposed framework outperforms the two mainstream hybrid sampling frameworks. Moreover, the proposed framework can effectively reduce the time consumption of hybrid sampling process.

M3S-ALG: Improved and robust prediction of allergenicity of chemical compounds by using a novel multi-step stacking strategy

A wide variety of chemicals cannot be introduced to the marketplace because of their high allergenicity. Therefore, it is fundamentally crucial to assess the allergenic potential of chemicals before introducing them into clinical therapeutics. However, assessing the allergenicity of chemical compounds experimentally is time-consuming and costly. To tackle this challenge, we propose M3S-ALG, a novel multi-step stacking strategy (M3S) for rapid and accurate identification of the allergenicity of chemical compounds by using only the SMILES notation. The proposed M3S method involves three steps, as follows. First, ten different balanced datasets were constructed using an under-sampling approach. Second, for each balanced dataset, 144 base-classifiers were trained and optimized to generate the prediction scores of allergenic chemical compounds considered as new probabilistic features. Third, we selected the important probabilistic features and employed them to construct the final stacked model (M3S-ALG). Experimental results show that M3S-ALG outperforms conventional ensemble strategies and its constituent base-classifiers on both the training and independent test datasets. This indicates the effectiveness and robustness of our proposed strategy in identifying the allergenicity of chemical compounds. In addition, M3S-ALG exhibited excellent prediction performance compared to existing methods on the independent test dataset, achieving a balanced accuracy of 0.877, MCC of 0.712, and AUC of 0.931. Finally, we developed a user-friendly online web server at https://pmlabqsar.pythonanywhere.com/M3SALG. This new approach is anticipated to facilitate the drug discovery and development community for the large-scale identification of chemical compounds with no allergenic properties.

A novel undersampling method based on data classification method

Data mining is one of the most important research area in literature. Due to the increasing volume of data, which is directly proportional to technological advancements, the number of researches in this field is growing rapidly. The goal of data mining is to extract various insights and obtain information from raw data by leveraging machine learning techniques. The structural characteristics and also class distributions of the datasets used in machine learning techniques significantly affect the performances of the algorithms. In this study, our aim is balancing the imbalanced binary dataset, used in the machine learning techniques, with an undersampling approach including a classification method via polyhedral conic functions.

Assessing the Performance of Machine Learning Models for Default Prediction under Missing Data and Class Imbalance: A Simulation Study

In the field of machine learning, robust model performance is essential for accurate predictions and informed decision-making. One critical challenge that hampers the effectiveness of machine learning algorithms is the presence of missing data. Missing values are ubiquitous in real-world datasets and can significantly impact the performance of predictive models. This study explores the impact of increasing levels of missing values on the performance of machine learning models. Simulated samples with missing values ranging from 5% to 50% were generated, and various models were evaluated accordingly. Missing data is a prevalent change that hinders the performance of machine learning algorithms. The results demonstrated a consistent trend of deteriorating model performance as the amount of missing values increases. Higher levels of missing values lead to decreased accuracy scores across all models. Among the models evaluated, decision trees (DT) and random forests (RF) consistently demonstrated high accuracy scores across all sampling techniques, showcasing their robustness in handling missing values. Logistic regression (LR) also performed relatively well, showing consistent performance across different levels of missing values. On the other hand, stochastic gradient descent classifier (SGDC), K-nearest neighbors (kNN), and naive Bayes (NB) models consistently exhibited lower accuracy scores across all sampling techniques, indicating limitations in handling missing values even when the dataset was more balanced. Furthermore, the study highlights the superiority of the SMOTE (Synthetic Minority OVER-sampling Technique) sampling technique compared to the UNDER-sampling approach. Models trained using SMOTE consistently achieved higher accuracy scores across all levels of missing values. This suggests that SMOTE sampling effectively handles imbalanced datasets and enhances classification performance, particularly when dealing with missing values. In an era where data fuels decision-making, this study's insights into the escalating impact of missing values on machine learning models stand as a clarion call for robust data handling techniques. As the quest for accurate predictions gains paramount importance, addressing the pervasive challenge of missing data emerges as a cornerstone for unlocking the true potential of machine learning in real-world applications.

Resampling and Ensemble Strategies for Churn Prediction

Churn analysis is a customer relationship management analytics that companies implement to predict the customers who are likely to terminate doing business with them. The success of marketing efforts to retain the existing customers is possible only if probable churners are correctly specified beforehand. Therefore, having powerful models with high prediction capabilities that lead to a profit growth is crucial. The imbalanced nature of churn datasets negatively effects the classification performance of machine learning methods. This study examines resampling –over- and under-sampling- and ensemble learning –bagging, boosting, and stacking– strategies integrated with the cross-validation procedure on imbalanced churn prediction. The experimental results, which are compared to the results of Support Vector Machines taken as the benchmark, show that ensemble methods improve the prediction performances. Also, applying over-sampling achieves a noticeable performance in comparison with the under-sampling approach.

Radial-based undersampling approach with adaptive undersampling ratio determination

Nowadays, machine learning techniques are employed in a wide range of applications, where classification is a common task in machine learning. It predicts the class label of a previously unseen example according to the decision of a classification model, which is learned by running a classifier learning algorithm on the collected training examples set. On the other hand, in many practical applications, the collected training sets are usually class imbalanced, that is, one class can have significantly more examples than the other class(es), but the minority class usually carries much valuable information and is more important than the majority class. However, most classifier learning algorithms are designed under the assumption that each class in a training set has approximately the same number of examples, leading to the consequence that they often can not achieve satisfactory classification performance on imbalanced data especially for the minority class examples. To solve this problem, a Radial-Based Undersampling approach with Adaptive undersampling Ratio (RBU-AR) is proposed in this paper. The main novelty of RBU-AR is that it attempts to determine the proper undersampling ratio according to the class overlap data complexity rather than adopting the default value 1 or using the empirical trial and error strategy as many existing undersampling approaches do. Experiments are conducted on 30 benchmark imbalanced datasets and 10 artificial datasets, the obtained results and corresponding statistical tests indicate that class overlap degree indeed has a great influence on the achievable classification performance and is usually more important than the class imbalance ratio IR, and our undersampling approach RBU-AR generally achieves highly competitive or better performance with respect to several state-of-the-art approaches. Therefore, this work provides a theoretical guideline in determining the proper extent of undersampling by utilizing the class overlap data complexity information.

Addressing the class-imbalance and class-overlap problems by a metaheuristic-based under-sampling approach

The problem of imbalanced class distribution in real-world datasets severely impairs the performance of classification algorithms. The learning task becomes more complicated and challenging when there is also the class-overlap problem in imbalanced data. This research tackles these problems by presenting an under-sampling approach based on a metaheuristic method in which the under-sampling problem is mapped into an optimization problem. The proposed approach aims to select an optimal subset of the majority samples to handle the imbalanced and the class-overlap problems simultaneously while avoiding the excessive elimination of majority samples, especially in overlapped regions. The quality of the generated solutions is evaluated by a classifier and optimized in an evolutionary process. Unlike most existing under-sampling methods, the majority samples are not removed only from the overlapped regions; the classifier performance determines the desired regions for eliminating the majority samples. Extensive experiments conducted on 66 synthetic and 24 real-world datasets with different imbalance ratios and overlapping degrees and two large high-dimensional datasets show a significant performance improvement from the proposed method compared to the competitors.

Multiclass classification of environmental chemical stimuli from unbalanced plant electrophysiological data.

Plant electrophysiological response contains useful signature of its environment and health which can be utilized using suitable statistical analysis for developing an inverse model to classify the stimulus applied to the plant. In this paper, we have presented a statistical analysis pipeline to tackle a multiclass environmental stimuli classification problem with unbalanced plant electrophysiological data. The objective here is to classify three different environmental chemical stimuli, using fifteen statistical features, extracted from the plant electrical signals and compare the performance of eight different classification algorithms. A comparison using reduced dimensional projection of the high dimensional features via principal component analysis (PCA) has also been presented. Since the experimental data is highly unbalanced due to varying length of the experiments, we employ a random under-sampling approach for the two majority classes to create an ensemble of confusion matrices to compare the classification performances. Along with this, three other multi-classification performance metrics commonly used for unbalanced data viz. balanced accuracy, F1-score and Matthews correlation coefficient have also been analyzed. From the stacked confusion matrices and the derived performance metrics, we choose the best feature-classifier setting in terms of the classification performances carried out in the original high dimensional vs. the reduced feature space, for this highly unbalanced multiclass problem of plant signal classification due to different chemical stress. Difference in the classification performances in the high vs. reduced dimensions are also quantified using the multivariate analysis of variance (MANOVA) hypothesis testing. Our findings have potential real-world applications in precision agriculture for exploring multiclass classification problems with highly unbalanced datasets, employing a combination of existing machine learning algorithms. This work also advances existing studies on environmental pollution level monitoring using plant electrophysiological data.

A Lightweight Double-Stage Scheme to Identify Malicious DNS over HTTPS Traffic Using a Hybrid Learning Approach

The Domain Name System (DNS) protocol essentially translates domain names to IP addresses, enabling browsers to load and utilize Internet resources. Despite its major role, DNS is vulnerable to various security loopholes that attackers have continually abused. Therefore, delivering secure DNS traffic has become challenging since attackers use advanced and fast malicious information-stealing approaches. To overcome DNS vulnerabilities, the DNS over HTTPS (DoH) protocol was introduced to improve the security of the DNS protocol by encrypting the DNS traffic and communicating it over a covert network channel. This paper proposes a lightweight, double-stage scheme to identify malicious DoH traffic using a hybrid learning approach. The system comprises two layers. At the first layer, the traffic is examined using random fine trees (RF) and identified as DoH traffic or non-DoH traffic. At the second layer, the DoH traffic is further investigated using Adaboost trees (ADT) and identified as benign DoH or malicious DoH. Specifically, the proposed system is lightweight since it works with the least number of features (using only six out of thirty-three features) selected using principal component analysis (PCA) and minimizes the number of samples produced using a random under-sampling (RUS) approach. The experiential evaluation reported a high-performance system with a predictive accuracy of 99.4% and 100% and a predictive overhead of 0.83 µs and 2.27 µs for layer one and layer two, respectively. Hence, the reported results are superior and surpass existing models, given that our proposed model uses only 18% of the feature set and 17% of the sample set, distributed in balanced classes.

Identification of Clinically Relevant HIV Vif Protein Motif Mutations through Machine Learning and Undersampling.

Human Immunodeficiency virus (HIV) and its clinical entity, the Acquired Immunodeficiency Syndrome (AIDS) continue to represent an important health burden worldwide. Although great advances have been made towards determining the way viral genetic diversity affects clinical outcome, genetic association studies have been hindered by the complexity of their interactions with the human host. This study provides an innovative approach for the identification and analysis of epidemiological associations between HIV Viral Infectivity Factor (Vif) protein mutations and four clinical endpoints (Viral load and CD4 T cell numbers at time of both clinical debut and on historical follow-up of patients. Furthermore, this study highlights an alternative approach to the analysis of imbalanced datasets, where patients without specific mutations outnumber those with mutations. Imbalanced datasets are still a challenge hindering the development of classification algorithms through machine learning. This research deals with Decision Trees, Naïve Bayes (NB), Support Vector Machines (SVMs), and Artificial Neural Networks (ANNs). This paper proposes a new methodology considering an undersampling approach to deal with imbalanced datasets and introduces two novel and differing approaches (MAREV-1 and MAREV-2). As theses approaches do not involve human pre-determined and hypothesis-driven combinations of motifs having functional or clinical relevance, they provide a unique opportunity to discover novel complex motif combinations of interest. Moreover, the motif combinations found can be analyzed through traditional statistical approaches avoiding statistical corrections for multiple tests.

Undersampling of approaching the classification boundary for imbalance problem

SummaryUsing imbalanced data in classification affect the accuracy. If the classification is based on imbalanced data directly, the results will have large deviations. A common approach to dealing with imbalanced data is to re‐structure the raw dataset via undersampling method. The undersampling method usually uses random or clustering approaches to trimming the majority class in the dataset, since some data in the majority class makes not contribute to classification model. In this paper a revised undersampling approach is proposed. First, we perform space compression in the vertical direction of the separating hyperplane. Then, a weighted random sampling hybrid ensemble learning method is carried out to make the sampled objects spread more widely near the separating hyperplane. Experiments with 7 under‐sampling methods on 21 imbalanced datasets show that our method has achieved good results.

Churn Prediction for High-Value Players in Freemium Mobile Games: Using Random Under-Sampling

Many game development companies use game data analysis for mining insights about users' behaviour and possible product growth. One of the most important analysis tasks for game development is user churn prediction. Effective churn prediction can help hold users in the game by initiating additional actions for their engagement. We focused on high-value user churn prediction as it is of particular interest for any business to keep paying customers satisfied and engaged. We consider the churn prediction problem as a classification problem and conduct the random undersampling approach to address imbalanced class distribution between churners and active users. Based on our real-life data from a freemium casual mobile game, although the best model was chosen as the final classification algorithm for extracted data, we can definitely say there is no general solution to the stated problem. Model performance highly depends on the churn definition, user segmentation and feature engineering, it is therefore necessary to have a custom approach to churn analysis in each specific case.

A machine learning framework to predict kidney graft failure with class imbalance using Red Deer algorithm

The prevalence of chronic kidney diseases has raised the demand for kidney replacement therapy over recent years. Although kidney transplantation provides better life quality than other alternatives, the replacement might face rejection and endanger individuals’ lives. Therefore, predicting this rejection is of great importance, and due to the complexity of the human body's immune system, it is a complicated task. Machine learning has been widely used in disease diagnoses such as graft rejection. One of the common issues in disease diagnosis is the class imbalance problem. The predictive model is likely to misclassify all of the minority class samples in such cases. This paper proposes a method that can accurately predict kidney rejection in an imbalanced dataset. We have adopted kidney transplantation data from 378 patients collected from 1994 to 2011. Our main contribution is to develop a novel clustering method using Red Deer Algorithm (RDA), which is later utilized in proposing a three-stage clustering-based undersampling approach to handle the class imbalance problem so that the final predictive models can classify the given data more accurately. The undersampling method includes denoising, RDA clustering, and sample selection. Moreover, the proposed clustering method is also used to reduce the data dimensionality. Subsequently, five different classification algorithms such as Support Vector Machine (SVM), Artificial Neural Network (ANN), K-Nearest Neighbor (KNN), Decision Tree (DT), and an ensemble method are used to predict kidney graft rejection. Then, these classifiers are compared in terms of five performance evaluation metrics: accuracy, sensitivity, specificity, F1 score, and area under the Receiver Operating Characteristic (ROC) curve (AUC). The obtained results indicate that the decision tree model outperformed other algorithms and achieved 0.96, 0.94, 0.97, 0.95, and 0.95 for accuracy, sensitivity, specificity, F1 score, and AUC, respectively. Hence, it is highly recommended to use the proposed method as a decision support system for clinical experts to predict kidney transplantation failure.

A Comparative Study on the Influence of Undersampling and Oversampling Techniques for the Classification of Physical Activities Using an Imbalanced Accelerometer Dataset.

Accelerometer data collected from wearable devices have recently been used to monitor physical activities (PAs) in daily life. While the intensity of PAs can be distinguished with a cut-off approach, it is important to discriminate different behaviors with similar accelerometry patterns to estimate energy expenditure. We aim to overcome the data imbalance problem that negatively affects machine learning-based PA classification by extracting well-defined features and applying undersampling and oversampling methods. We extracted various temporal, spectral, and nonlinear features from wrist-, hip-, and ankle-worn accelerometer data. Then, the influences of undersampilng and oversampling were compared using various ML and DL approaches. Among various ML and DL models, ensemble methods including random forest (RF) and adaptive boosting (AdaBoost) exhibited great performance in differentiating sedentary behavior (driving) and three walking types (walking on level ground, ascending stairs, and descending stairs) even in a cross-subject paradigm. The undersampling approach, which has a low computational cost, exhibited classification results unbiased to the majority class. In addition, we found that RF could automatically select relevant features for PA classification depending on the sensor location by examining the importance of each node in multiple decision trees (DTs). This study proposes that ensemble learning using well-defined feature sets combined with the undersampling approach is robust for imbalanced datasets in PA classification. This approach will be useful for PA classification in the free-living situation, where data imbalance problems between classes are common.

High-Resolution Free-Breathing Quantitative First-Pass Perfusion Cardiac MR Using Dual-Echo Dixon With Spatio-Temporal Acceleration.

IntroductionTo develop and test the feasibility of free-breathing (FB), high-resolution quantitative first-pass perfusion cardiac MR (FPP-CMR) using dual-echo Dixon (FOSTERS; Fat-water separation for mOtion-corrected Spatio-TEmporally accelerated myocardial peRfuSion).Materials and MethodsFOSTERS was performed in FB using a dual-saturation single-bolus acquisition with dual-echo Dixon and a dynamically variable Cartesian k-t undersampling (8-fold) approach, with low-rank and sparsity constrained reconstruction, to achieve high-resolution FPP-CMR images. FOSTERS also included automatic in-plane motion estimation and T correction to obtain quantitative myocardial blood flow (MBF) maps. High-resolution (1.6 x 1.6 mm2) FB FOSTERS was evaluated in eleven patients, during rest, against standard-resolution (2.6 x 2.6 mm2) 2-fold SENSE-accelerated breath-hold (BH) FPP-CMR. In addition, MBF was computed for FOSTERS and spatial wavelet-based compressed sensing (CS) reconstruction. Two cardiologists scored the image quality (IQ) of FOSTERS, CS, and standard BH FPP-CMR images using a 4-point scale (1–4, non-diagnostic – fully diagnostic).ResultsFOSTERS produced high-quality images without dark-rim and with reduced motion-related artifacts, using an 8x accelerated FB acquisition. FOSTERS and standard BH FPP-CMR exhibited excellent IQ with an average score of 3.5 ± 0.6 and 3.4 ± 0.6 (no statistical difference, p > 0.05), respectively. CS images exhibited severe artifacts and high levels of noise, resulting in an average IQ score of 2.9 ± 0.5. MBF values obtained with FOSTERS presented a lower variance than those obtained with CS.DiscussionFOSTERS enabled high-resolution FB FPP-CMR with MBF quantification. Combining motion correction with a low-rank and sparsity-constrained reconstruction results in excellent image quality.

A Salp Swarm-Based Under-Sampling Approach for Medical Imbalanced Data Classification

Data imbalance refers to the unequal distribution of classes within a dataset that directly affects the accuracy of machine learning classification algorithms. Although many resampling techniques have been proposed by researchers, learning from imbalanced data is still considered one of the contemporary challenges. The class imbalanced problem has been complicated as most of the existing techniques don't manage the similarity relationships between minority and majority classes well. In addition, due to the complex relationships among classes, most of the existing techniques do not focus on retaining valuable samples in the majority class(es) properly. In this article, a salp swarm optimization-based under-sampling technique (SSBUT) is proposed to address data class imbalance problems. Utilizing the proposed SSBUT, the similarity relationship among the samples of the majority class is well analyzed, and the samples that do not affect the accuracy of the classification algorithm are eliminated from the majority class. The performance of the proposed SSBUT has been tested on benchmark medical imbalanced datasets and the obtained results have been compared with state-of-the-art under-sampling techniques. The experimental results show that the proposed SSBUT consistently outperformed the state-of-the-art under-sampling techniques in terms of various evaluation criteria.

Supervised Machine Learning Approaches on Multispectral Remote Sensing Data for a Combined Detection of Fire and Burned Area

Bushfires pose a severe risk, among others, to humans, wildlife, and infrastructures. Rapid detection of fires is crucial for fire-extinguishing activities and rescue missions. Besides, mapping burned areas also supports evacuation and accessibility to emergency facilities. In this study, we propose a generic approach for detecting fires and burned areas based on machine learning (ML) approaches and remote sensing data. While most studies investigated either the detection of fires or mapping burned areas, we addressed and evaluated, in particular, the combined detection on three selected case study regions. Multispectral Sentinel-2 images represent the input data for the supervised ML models. First, we generated the reference data for the three target classes, burned, unburned, and fire, since no reference data were available. Second, the three regional fire datasets were preprocessed and divided into training, validation, and test subsets according to a defined schema. Furthermore, an undersampling approach ensured the balancing of the datasets. Third, seven selected supervised classification approaches were used and evaluated, including tree-based models, a self-organizing map, an artificial neural network, and a one-dimensional convolutional neural network (1D-CNN). All selected ML approaches achieved satisfying classification results. Moreover, they performed a highly accurate fire detection, while separating burned and unburned areas was slightly more challenging. The 1D-CNN and extremely randomized tree were the best-performing models with an overall accuracy score of 98% on the test subsets. Even on an unknown test dataset, the 1D-CNN achieved high classification accuracies. This generalization is even more valuable for any use-case scenario, including the organization of fire-fighting activities or civil protection. The proposed combined detection could be extended and enhanced with crowdsourced data in further studies.

Deep transfer learning benchmark for plastic waste classification

Millions of people throughout the world have been harmed by plastic pollution. There are microscopic pieces of plastic in the food we eat, the water we drink, and even the air we breathe. Every year, the average human consumes 74,000 microplastics, which has a significant impact on their health. This pollution must be addressed before it has a significant negative influence on the population. This research benchmarks six state-of-the-art convolutional neural network models pre-trained on the ImageNet Dataset. The models Resnet-50, ResNeXt, MobileNet_v2, DenseNet, SchuffleNet and AlexNet were tested and evaluated on the WaDaBa plastic dataset, to classify plastic types based on their resin codes by integrating the power of transfer learning. The accuracy and training time for each model has been compared in this research. Due to the imbalance in the data, the under-sampling approach has been used. The ResNeXt model attains the highest accuracy in fourteen minutes.

Learning automata based particle swarm optimization for solving class imbalance problem

Class imbalance is an important problem in many domains such as disease classification, network intrusion detection, fraud detection, and spam filtering. While dealing with imbalanced datasets, traditional supervised machine learning algorithms do not often provide acceptable results. Several approaches are used to handle the class imbalance problem. Of these, undersampling approaches are mostly followed by the researchers in which the number of instances in the majority class gets reduced. Selection of instances from the majority class can be considered as an optimization problem. To this end, in this paper, we present an undersampling approach based on widely-used Particle Swarm Optimization (PSO). The majority class samples are first clustered to form the initial undersampled set. The samples to be selected are then optimized using PSO to give the best model. The parameters of PSO are fine tuned using Learning Automata. Appropriate metrics suitable for class imbalance problems have been used to construct the fitness function for optimizing the undersampled training set. The proposed method has achieved 2% to 10% performance improvement over most of the contemporary methods on various datasets with imbalance ratios ranging from 5 to 130, thus showing that the method is robust and useful in practical scenarios. The code of the proposed method can be accessed via https://github.com/kkg1999/Undersampling.

A New Hybrid Under-sampling Approach to Imbalanced Classification Problems

ABSTRACT Among many machine learning applications, classification is one of the important tasks. Most classification algorithms have been designed under the assumption that the number of samples for each class is approximately balanced. However, if the conventional classification approaches are applied to a class imbalanced dataset, it is likely to cause misclassification and, as a result, may distort classification performance results. Thus, in this study, we consider imbalanced classification problems and adopt an efficient preprocessing technique to improve the classification performances. In particular, we focus on borderline noise and outlier samples that belong to the majority class since they may influence classification performance. For this, we propose a hybrid resampling method, called BOD-based under-sampling, which is based on density-based spatial clustering of applications with noise (DBSCAN) approach as well as noise and outlier detection methods, that is, borderline noise factor (BNF) and outlierness based on neighborhood (OBN) to divide majority class samples into four distinctive categories, i.e., safe, borderline noise, rare, and outlier. Specifically, we first determine the borderline noise samples in the overlapped region using the BNF method. Secondly, we use the OBN method to detect outlier samples and apply the DBSCAN approach to cluster the samples. Based on the results obtained from the sample identification analysis, we then segregate the safe category samples which are not abnormal samples while keeping the rest of the samples as rare samples. Finally, we remove some of safe samples by using the random under-sampling (RUS) method and verify the effectiveness of the proposed algorithm through the comprehensive experimental analysis with considering several class imbalance datasets.