Oversampling Method Research Articles

A Novel Oversampling Method for Imbalanced Datasets Based on Density Peaks Clustering

A Novel Oversampling Method for Imbalanced Datasets Based on Density Peaks Clustering

Optimization of software defects prediction in imbalanced class using a combination of resampling methods with support vector machine and logistic regression

The main problem in producing high accuracy software defect prediction is if the data set has an imbalance class and dichotomous characteristics. The imbalanced class problem can be solved using a data level approach, such as resampling methods. While the problem of software defects predicting if the data set has dichotomous characteristics can be approached using the classification method. This study aimed to analyze the performance of the proposed software defect prediction method to identify the best combination of resampling methods with the appropriate classification method to provide the highest accuracy. The combination of the proposed methods first is the resampling process using oversampling, under-sampling, or hybrid methods. The second process uses the classification method, namely the Support Vector Machine (SVM) algorithm and the Logistic Regression (LR) algorithm. The proposed, tested model uses five NASA MDP data sets with the same number attributes of 37. Based on the t-test, the < = 0.0344 < 0.05 and the > = 3.1524 > 2.7765 which indicates that the combination of the proposed methods is suitable for classifying imbalanced class. The performance of the classification algorithm has also improved with the use of the resampling process. The average increase in AUC values using the resampling in the SVM algorithm is 17.19%, and the LR algorithm is at 7.26% compared to without the resampling process. Combining the three resampling methods with the SVM algorithm and the LR algorithm shows that the best combining method is the oversampling method with the SVM algorithm to software defects prediction in imbalanced class with an average accuracy value of 84.02% and AUC 91.65%.

Distance-based Probabilistic Data Augmentation for Synthetic Minority Oversampling

Class imbalance can adversely affect the performance of machine learning for prediction and classification. One approach to address the class imbalance problem is synthetic minority oversampling. Oversampling approaches can be broadly categorized as either being structural or statistical in nature. Structural approaches generally have the advantage of identifying and oversampling those minority data points that best facilitate class separation, while statistical approaches model the underlying distribution from which the minority samples can be drawn. In this article, we formulate a distance-based approach that generates samples by both modeling the underlying minority class distribution and by geometrically considering those borderline samples entangled in the majority class. We demonstrate the efficacy of our approach operating on the Class-Imbalance data set from UCI by comparing its mean accuracy, AUC and F 1 -score performance against both statistical and structural synthetic minority oversampling methods.

Optimization and comparison of models for core temperature prediction of mother rabbits using infrared thermography

Core temperature is a measurement used to evaluate and diagnose the health status of mother rabbits. The preferred method is to use an indwelling probe to measure the rectal temperature. This procedure can be stressful to rabbits, are time and labor-consuming. The objective of this paper was to develop the fastest, accurate and non-invasive method to measure the rectal temperature of mother rabbits using Infrared thermography (IRT) and machine learning (ML). Two hundred lactating mother rabbits (LR) and two hundred non-lactating mother rabbits (NLR) of the YPLU breed were used in the experiment and IRT data, rectal temperature and ambient temperature were recorded. The results showed that there was a significant correlation between the maximum infrared temperature of eyes (EMIT) and ears (RMIT) and rectal temperature (p < 0.05) and a significant correlation between the EMIT, RMIT and the ambient temperature (p < 0.01) in two kinds of mother rabbits. Least squares support vector machine (LS-SVM), Gaussian process (GP), and partial least squares (PLS) machine learning methods were used to establish models of mother rabbits’ core temperature. Using the over-sampling method to process the training set, three models of LS-SVM, GP, and PLS with improved generalization performance were obtained. The RMSE for the test set by LS-SVM, GP, and PLS methods were 0.13℃, 0.12℃ and 0.07℃ and the R2 were 0.94, 0.96 and 0.98, respectively. Compared with the results obtained from two other regressions, LS-SVM and GP, the PLS method exhibited the best performance. The prediction error of the test set is <0.2℃, which can better predict the core temperature of the mother rabbits.

Binary imbalanced data classification based on diversity oversampling by generative models

In many practical applications, the data are class imbalanced. Accordingly, it is very meaningful and valuable to investigate the classification of imbalanced data. In the framework of binary imbalanced data classification, the synthetic minority oversampling technique (SMOTE) is the best-known oversampling method. However, for each positive sample, SMOTE generates only k synthetic samples on the lines between the positive sample and its k-nearest neighbors, resulting in three drawbacks: (1) SMOTE cannot effectively extend the training field of positive samples; (2) the generated positive samples lack diversity; (3) SMOTE does not accurately approximate the probability distribution of the positive samples. Therefore, two binary imbalanced data classification methods named BIDC1 and BIDC2 based on diversity oversampling by generative models are proposed. The BIDC1 and BIDC2 conduct diversity oversampling using extreme learning machine autoencoder and generative adversarial network, respectively. Extensive experiments on 26 data sets are conducted to compare the two methods with 14 state-of-the-art methods using five metrics: F-measure, G-means, AUC-area, MMD-score, and Silhouette-score. The experimental results demonstrate that the two methods outperform the other 14 methods.

Development of glaucoma predictive model and risk factors assessment based on supervised models

ObjectivesTo develop and to propose a machine learning model for predicting glaucoma and identifying its risk factors.MethodData analysis pipeline is designed for this study based on Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology. The main steps of the pipeline include data sampling, preprocessing, classification and evaluation and validation. Data sampling for providing the training dataset was performed with balanced sampling based on over-sampling and under-sampling methods. Data preprocessing steps were missing value imputation and normalization. For classification step, several machine learning models were designed for predicting glaucoma including Decision Trees (DTs), K-Nearest Neighbors (K-NN), Support Vector Machines (SVM), Random Forests (RFs), Extra Trees (ETs) and Bagging Ensemble methods. Moreover, in the classification step, a novel stacking ensemble model is designed and proposed using the superior classifiers.ResultsThe data were from Shahroud Eye Cohort Study including demographic and ophthalmology data for 5190 participants aged 40-64 living in Shahroud, northeast Iran. The main variables considered in this dataset were 67 demographics, ophthalmologic, optometric, perimetry, and biometry features for 4561 people, including 4474 non-glaucoma participants and 87 glaucoma patients. Experimental results show that DTs and RFs trained based on under-sampling of the training dataset have superior performance for predicting glaucoma than the compared single classifiers and bagging ensemble methods with the average accuracy of 87.61 and 88.87, the sensitivity of 73.80 and 72.35, specificity of 87.88 and 89.10 and area under the curve (AUC) of 91.04 and 94.53, respectively. The proposed stacking ensemble has an average accuracy of 83.56, a sensitivity of 82.21, a specificity of 81.32, and an AUC of 88.54.ConclusionsIn this study, a machine learning model is proposed and developed to predict glaucoma disease among persons aged 40-64. Top predictors in this study considered features for discriminating and predicting non-glaucoma persons from glaucoma patients include the number of the visual field detect on perimetry, vertical cup to disk ratio, white to white diameter, systolic blood pressure, pupil barycenter on Y coordinate, age, and axial length.

LDAS: Local density-based adaptive sampling for imbalanced data classification

Class imbalance poses a great challenge to traditional classifiers in machine learning as they strongly favor the majority class while ignoring the minority class. Synthetic over-sampling methods deal with this problem by generating synthetic examples to balance the distribution of data. However, most existing methods prefer to generate synthetic examples in a specific area without considering the complexity of imbalance distribution, which may result in the over-emphasis of learning model on some data difficulty factors. To this end, we propose a local density-based adaptive sampling method (LDAS) for imbalanced data. LDAS first assigns a local density for each minority example, then a new cleaning strategy is proposed to remove the overlapping majority examples. Finally, it weighs each minority example based on its approaching degree of decision boundary and the corresponding local density. This is done in such a way that synthetic examples are generated in the safe area and the border area simultaneously according to the weight of minority examples. Extensive experiments on KEEL datasets demonstrate the effectiveness of the proposal LDAS.

An oversampling method for multi-class imbalanced data based on composite weights.

To solve the oversampling problem of multi-class small samples and to improve their classification accuracy, we develop an oversampling method based on classification ranking and weight setting. The designed oversampling algorithm sorts the data within each class of dataset according to the distance from original data to the hyperplane. Furthermore, iterative sampling is performed within the class and inter-class sampling is adopted at the boundaries of adjacent classes according to the sampling weight composed of data density and data sorting. Finally, information assignment is performed on all newly generated sampling data. The training and testing experiments of the algorithm are conducted by using the UCI imbalanced datasets, and the established composite metrics are used to evaluate the performance of the proposed algorithm and other algorithms in comprehensive evaluation method. The results show that the proposed algorithm makes the multi-class imbalanced data balanced in terms of quantity, and the newly generated data maintain the distribution characteristics and information properties of the original samples. Moreover, compared with other algorithms such as SMOTE and SVMOM, the proposed algorithm has reached a higher classification accuracy of about 90%. It is concluded that this algorithm has high practicability and general characteristics for imbalanced multi-class samples.

An improved learning-based LSTM approach for lane change intention prediction subject to imbalanced data

Lane change intention prediction is an essential component for motion planning of Autonomous Vehicles (AVs). In this work, we aim to achieve this task by using Long-Short Term Memory (LSTM) network. One critical challenge on this task is that the dataset used for training such a network is usually highly imbalanced due to the fact that the size of left/right lane change data is much smaller than that of the lane keeping data. The imbalanced dateset would lead to trivial output of LSTM model. To deal with this problem, we propose a hierarchical over-sampling bagging method based on Grey Wolf Optimizer (GWO) algorithm and Synthetic Minority Over-sampling Technique (SMOTE). With the proposed method, more diverse and informative instances of minority classes can be generated for training LSTM model. Furthermore, we also propose a sampling technique to keep the temporal information and make the proposed method applicable to sequential data. Moreover, to further improve the prediction performance, we also take the interactions between neighboring vehicles into account by concatenating their trajectories when constructing features. We evaluate our method against several baseline algorithms over two benchmark datasets and the empirical results validate the effectiveness and efficiency of our method in terms of the indexes of prediction time, F1, and G-mean.

Exploring ensemble oversampling method for imbalanced keyword extraction learning in policy text based on three-way decisions and SMOTE

The e-government platform not only enables the government department to publish policy texts online, but also makes it easier for users to access the policy, especially for the convenience of understanding the policies by reading the keywords. For a given policy text, keywords take up only a small proportion, which can be seen as an unbalanced data set. Therefore, in this paper, we try to design automatic keyword extraction method of policy text with unbalanced data set. In order to achieve this goal, we firstly propose a new ensemble oversampling method to synthesize new data. In this case, we sample data from the training set by using Bagging method. During each sampling process, we train a logistic regression model to classify the training set. Based on the predicted probabilities, we utilize the classification confidence to divide training set into three regions by using three-way decisions (3WD). Then, we implement different strategies to synthesize new data. Besides, for keyword extraction of policy text, we conduct a series of experiments by using the classical supervised and unsupervised methods. In our experiment results, we can find that both in the public data sets and manual data sets, our sampling method can achieve better performance of F-measure and G-mean indexes, no matter what the supervised machine learning method is. This can also explain the advantage of 3WD. Different regions have different strategies to synthesize new data.

A new instance density-based synthetic minority oversampling method for imbalanced classification problems

Many useful data pre-processing techniques have been developed to improve the performance of imbalanced classification problems. However, the results from the adoption of the several pre-processing methods indicated that the class imbalance is not the only reason for difficulties in learning in such a situation but the pre-processing for specific minority classes can improve classification performance more effectively. Thus, in this research, we present a new synthetic minority oversampling method based on instance density to enhance the performance of imbalanced classification problems. For this, two dimensional reduction techniques, i.e., principal component analysis (PCA) and factor analysis of mixed data (FAMD), are used to map the instances and k-nearest neighbor (kNN) method is used to classify minority class instances into four distinct types. Finally, through the comprehensive experiments on several imbalance datasets with different combination of subcategories, it is shown that the proposed method outperform the traditional oversampling methods.

Performance Analysis of Two-Stage Iterative Ensemble Method over Random Oversampling Methods on Multiclass Imbalanced Datasets

Data imbalance occurring among multiclass datasets is very common in real-world applications. Existing studies reveal that various attempts were made in the past to overcome this multiclass imbalance problem, which is a severe issue related to the typical supervised machine learning methods such as classification and regression. But, still there exists a need to handle the imbalance problem efficiently as the datasets include both safe and unsafe minority samples. Most of the widely used oversampling techniques like SMOTE and its variants face challenges in replicating or generating the new data instances for balancing them across multiple classes, particularly when the imbalance is high and the number of rare samples count is too minimal thus leading the classifier to misclassify the data instances. To lessen this problem, we proposed a new data balancing method namely a two-stage iterative ensemble method to tackle the imbalance in multiclass environment. The proposed approach focuses on the rare minority sample’s influence on learning from imbalanced datasets and the main idea of the proposed approach is to balance the data without any change in class distribution before it gets trained by the learner such that it improves the learner’s learning process. Also, the proposed approach is compared against two widely used oversampling techniques and the results reveals that the proposed approach shows a much significant improvement in the learning process among the multiclass imbalanced data.

Evolutionary Mahalanobis Distance-Based Oversampling for Multi-Class Imbalanced Data Classification.

The number of sensing data are often imbalanced across data classes, for which oversampling on the minority class is an effective remedy. In this paper, an effective oversampling method called evolutionary Mahalanobis distance oversampling (EMDO) is proposed for multi-class imbalanced data classification. EMDO utilizes a set of ellipsoids to approximate the decision regions of the minority class. Furthermore, multi-objective particle swarm optimization (MOPSO) is integrated with the Gustafson–Kessel algorithm in EMDO to learn the size, center, and orientation of every ellipsoid. Synthetic minority samples are generated based on Mahalanobis distance within every ellipsoid. The number of synthetic minority samples generated by EMDO in every ellipsoid is determined based on the density of minority samples in every ellipsoid. The results of computer simulations conducted herein indicate that EMDO outperforms most of the widely used oversampling schemes.

Oversampling the minority class using a dedicated fitness function and genetic algorithmic progression

SummaryClass imbalance is a pertinent characteristic of a number of real‐world datasets. The underrepresentation of the minority class in such datasets causes difficulty in learning and prediction of its members. In this era of Big Data, we need to address this practical issue in order to get admissible outputs from our classifier models. We present a novel work, oversampling the minority class using a dedicated fitness function and genetic algorithmic progression – to tackle the problem of class imbalance. We generate a random set of feature points in the given space and use genetic‐algorithm guided progression to metamorphose them into a set of potential minority points. The optimization function looks at the neighborhood of the synthetic minority points for computing the fitness scores. We have employed 19 real‐world datasets, three metrics, three classifiers, and four diversified oversampling methods in the empirical study. A comprehensive study involving several well‐established minority oversamplers demonstrates the efficaciousness of the proposed method in handling class imbalance.

A Safe Zone SMOTE Oversampling Algorithm Used in Earthquake Prediction Based on Extreme Imbalanced Precursor Data

Earthquake prediction based on extreme imbalanced precursor data is a challenging task for standard algorithms. Since even if an area is in an earthquake-prone zone, the proportion of days with earthquakes per year is still a minority. The general method is to generate more artificial data for the minority class that is the earthquake occurrence data. But the most popular oversampling methods generate synthetic samples along line segments that join minority class instances, which is not suitable for earthquake precursor data. In this paper, we propose a Safe Zone Synthetic Minority Oversampling Technique (SZ-SMOTE) oversampling method as an enhancement of the SMOTE data generation mechanism. SZ-SMOTE generates synthetic samples with a concentration mechanism in the hyper-sphere area around each selected minority instances. The performance of SZ-SMOTE is compared against no oversampling, SMOTE and its popular modifications adaptive synthetic sampling (ADASYN) and borderline SMOTE (B-SMOTE) on six different classifiers. The experiment results show that the quality of earthquake prediction using SZ-SMOTE as oversampling algorithm significantly outperforms that of using the other oversampling algorithms.

Improvising Balancing Methods for Classifying Imbalanced Data

Abstract: Class Imbalance problem is one of the most challenging problems faced by the machine learning community. As we refer the imbalance to various instances in class of being relatively low as compare to other data. A number of over - sampling and under-sampling approaches have been applied in an attempt to balance the classes. This study provides an overview of the issue of class imbalance and attempts to examine various balancing methods for dealing with this problem. In order to illustrate the differences, an experiment is conducted using multiple simulated data sets for comparing the performance of these oversampling methods on different classifiers based on various evaluation criteria. In addition, the effect of different parameters, such as number of features and imbalance ratio, on the classifier performance is also evaluated. Keywords: Imbalanced learning, Over-sampling methods, Under-sampling methods, Classifier performances, Evaluationmetrices

Anovel HEOMGA Approach for Class Imbalance Problem in the Application of Customer Churn Prediction

Making class balance is essential when learning from highly skewed datasets; otherwise, a learner may classify all instances to a negative class, resulting in a high false-negative rate. As a result, a precise balancing strategy is required. Many researchers have investigated class imbalance using Machine Learning (ML) methods due to their powerful generalization performance and interpreting capabilities, comparing with random sampling techniques, to handle the problem of class imbalance in the preprocessing phase to facilitate learning process and improve performance results of learners. In this research, an effective method called HEOMGA is presented by combining Heterogeneous Euclidean-Overlap Metric (HEOM) and Genetic Algorithm (GA) for oversampling minority class. The HEOM is employed to define a fitness function for the GA. To assess the performance of the proposed HEOMGA method, three benchmark datasets from UCI repository in the domain of customer churn prediction are examined using three different ML learners and evaluated with three performance metrics. The experiment results show the effectiveness of the proposed method compared to some popular oversample methods, such as SMOTE, ADASYN, G SMOTE, and Gaussian oversampling methods. The HEOMGA method significantly outperformed the other oversampling methods in terms of recall, G mean, and AUC when the Wilcoxon signed-rank test is used.

Improvement of the Classification Performance of an Intrusion Detection Model for Rare and Unknown Attack Traffic

How to deal with rare and unknown data in traffic classification has a decisive influence on classification performance. Rare data make it difficult to generate validation datasets to prevent overfitting, and unknown data interferes with learning and degrades the performance of the model. This paper presents a model generation method that accurately classifies rare data and new types of attacks, and does not result in overfitting. First, we use oversampling methods to solve the data imbalance caused by rare data. We separate the test dataset into a training dataset and a validation dataset. A model is created using separate training and validation datasets. Furthermore, the test dataset is used only for evaluating the performance capabilities of classification models, in order to make the test dataset independent of learning. We also use a softmax function that numerically indicates the probability that the model’s predictive results are accurate in detecting new, unknown attacks. Consequently, when applying the proposed method to the NSL_KDD dataset, the accuracy is 91.66%—an improvement of 6–16% compared to existing methods.

Short-Term Metro Ridership Prediction During Unplanned Events

Unplanned events present significant challenges for operations and management in metro systems. Short-term ridership prediction can help agencies to better design contingency strategies under unplanned events. Though many short-term prediction methods have been proposed in the literature, most studies focused on typical situations or planned events. The study develops methods for the short-term metro ridership prediction under unplanned events. It explores event impact representation mechanisms and deals with the imbalanced data training problem in building the prediction model under unplanned events. Typical machine learning and deep learning methods are developed for exploration. A large-scale automatic fare collection (AFC) dataset and event record data for a heavily used metro system are used for empirical studies. The analysis found that the same type of unplanned event shares a similar and consistent demand change pattern (with respect to the demand under typical situations) at the station level. The synthetic minority oversampling technique (SMOTE) can enrich the ridership observations under unplanned events and generate a balanced dataset for model training. Given the occurrence of unplanned events, the results show that a combination of demand change ratio and the SMOTE oversampling technique enables the prediction models to learn the impact of unplanned events and improve the prediction accuracy under unplanned events. However, the oversampling methods (i.e., SMOTE and replication) slightly deteriorate the prediction accuracy for ridership under normal conditions. The findings provide insights into mechanisms for disruption impact representation and oversampling imbalanced data in model training, and guide the development of models for short-term prediction under unplanned events.

Comparing Machine Learning Methods to Improve Fall Risk Detection in Elderly with Osteoporosis from Balance Data.

Falls are a multifactorial cause of injuries for older people. Subjects with osteoporosis are particularly vulnerable to falls. We study the performance of different computational methods to identify people with osteoporosis who experience a fall by analysing balance parameters. Balance parameters, from eyes open and closed posturographic studies, and prospective registration of falls were obtained from a sample of 126 community-dwelling older women with osteoporosis (age 74.3 ± 6.3) using World Health Organization Questionnaire for the study of falls during a follow-up of 2.5 years. We analyzed model performance to determine falls of every developed model and to validate the relevance of the selected parameter sets. The principal findings of this research were (1) models built using oversampling methods with either IBk (KNN) or Random Forest classifier can be considered good options for a predictive clinical test and (2) feature selection for minority class (FSMC) method selected previously unnoticed balance parameters, which implies that intelligent computing methods can extract useful information with attributes which otherwise are disregarded by experts. Finally, the results obtained suggest that Random Forest classifier using the oversampling method to balance the data independent of the set of variables used got the best overall performance in measures of sensitivity (>0.71), specificity (>0.18), positive predictive value (PPV >0.74), and negative predictive value (NPV >0.66) independent of the set of variables used. Although the IBk classifier was built with oversampling data considering information from both eyes opened and closed, using all variables got the best performance (sensitivity >0.81, specificity >0.19, PPV = 0.97, and NPV = 0.66).