Existing Oversampling Methods Research Articles

Deep Representation Learning With Sample Generation and Augmented Attention Module for Imbalanced ECG Classification.

Developing an efficient heartbeat monitoring system has become a focal point in numerous healthcare applications. Specifically, in the last few years, heartbeat classification for arrhythmia detection has gained considerable interest from researchers. This paper presents a novel deep representation learning method for the efficient detection of arrhythmic beats. To mitigate the issues associated with the imbalanced data distribution, a novel re-sampling strategy is introduced. Unlike the existing oversampling methods, the proposed technique transforms majority-class samples into minority-class samples with a novel translation loss function. This approach assists the model in learning a more generalized representation of crucially important minority class samples. Moreover, by exploiting an auxiliary feature, an augmented attention module is designed that focuses on the most relevant and target-specific information. We adopted an inter-patient classification paradigm to evaluate the proposed method. The experimental results of this study on the MIT-BIH arrhythmia database clearly indicate that the proposed model with augmented attention mechanism and over-sampling strategy significantly learns a balanced deep representation and improves the classification performance of vital heartbeats.

A quantum-based oversampling method for classification of highly imbalanced and overlapped data.

Data imbalance is a challenging problem in classification tasks, and when combined with class overlapping, it further deteriorates classification performance. However, existing studies have rarely addressed both issues simultaneously. In this article, we propose a novel quantum-based oversampling method (QOSM) to effectively tackle data imbalance and class overlapping, thereby improving classification performance. QOSM utilizes the quantum potential theory to calculate the potential energy of each sample and selects the sample with the lowest potential as the center of each cover generated by a constructive covering algorithm. This approach optimizes cover center selection and better captures the distribution of the original samples, particularly in the overlapping regions. In addition, oversampling is performed on the samples of the minority class covers to mitigate the imbalance ratio (IR). We evaluated QOSM using three traditional classifiers (support vector machines [SVM], k-nearest neighbor [KNN], and naive Bayes [NB] classifier) on 10 publicly available KEEL data sets characterized by high IRs and varying degrees of overlap. Experimental results demonstrate that QOSM significantly improves classification accuracy compared to approaches that do not address class imbalance and overlapping. Moreover, QOSM consistently outperforms existing oversampling methods tested. With its compatibility with different classifiers, QOSM exhibits promising potential to improve the classification performance of highly imbalanced and overlapped data.

Label correlation guided borderline oversampling for imbalanced multi-label data learning

Multi-label data classification has received much attention due to its wide range of application domains. Unfortunately, a class imbalance problem often occurs in multi-label datasets, causing challenges for classification algorithms. Oversampling is one of the most important approaches, as it generates minority label instances to balance the class distribution. However, existing oversampling methods ignore existing label correlations, resulting in the generation of inappropriate synthetic minority samples and making multi-label data classification tasks harder. In this work, we propose an oversampling method that considers label correlations and identifies two critical boundary regions for generating synthetic minority samples. Moreover, we propose a weighting strategy to assign weights to these instances based on their distance information. To evaluate the performance of our proposed method, we conducted experiments on sixteen public datasets. The results show that our approach outperforms the state-of-the-art approaches in terms of various assessment metrics, such as Macro F1 and Macro AUC. The code is available at https://github.com/IntelliDAL/Multi-label/tree/main/LCOS.

NanBDOS: Adaptive and parameter-free borderline oversampling via natural neighbor search for class-imbalance learning

Learning class-imbalance data has become a challenging task in machine learning. Oversampling is an effective way to achieve rebalancing between classes by generating new minority instances. However, most existing oversampling methods exhibit significant ill-posedness due to the involvement of k-nearest neighbors. Recently, several novel techniques have been developed using natural neighbors instead of k-nearest neighbors. But these budding techniques fail to take into account the natural distribution of data, resulting in degraded adaptability of natural neighbors to data characteristics. In this paper, we propose an adaptive and completely parameter-free synthetic oversampling method called NanBDOS (short for borderline oversampling via natural neighbor search). NanBDOS first determines the intrinsic neighbor relationship between instances through a natural neighbor search procedure. Then, those informative borderline instances are identified and used as base instances, i.e., sampling seeds. NanBDOS assigns dynamic sampling weights to the base instances so that the data complexity can be well represented. Finally, new synthetic instances are generated by interpolating between the base instances and their natural neighbors. NanBDOS is experimentally compared with seven baseline methods on twenty-four real-world datasets. The results confirm the effectiveness of the proposed method. Moreover, the statistical analysis also indicates its higher-level Friedman ranking.

An oversampling method for imbalanced data based on spatial distribution of minority samples SD-KMSMOTE

With the rapid expansion of data, the problem of data imbalance has become increasingly prominent in the fields of medical treatment, finance, network, etc. And it is typically solved using the oversampling method. However, most existing oversampling methods randomly sample or sample only for a particular area, which affects the classification results. To solve the above limitations, this study proposes an imbalanced data oversampling method, SD-KMSMOTE, based on the spatial distribution of minority samples. A filter noise pre-treatment is added, the category information of the near-neighbouring samples is considered, and the existing minority class sample noise is removed. These conditions lead to the design of a new sample synthesis method, and the rules for calculating the weight values are constructed on this basis. The spatial distribution of minority class samples is considered comprehensively; they are clustered, and the sub-clusters that contain useful information are assigned larger weight values and more synthetic sample numbers. The experimental results show that the experimental results outperform existing methods in terms of precision, recall, F1 score, G-mean, and area under the curve values when the proposed method is used to expand the imbalanced dataset in the field of medicine and other fields.

Grouping-based Oversampling in Kernel Space for Imbalanced Data Classification

The class-imbalanced classification is a difficult problem because not only traditional classifiers are more biased towards the majority classes and inclined to generate incorrect predictions, but also the existing algorithms often have difficulty tackling this kind of problem with the class overlapping. Oversampling is a widely used and effective method to obtain balanced samples for imbalanced data, but the existing oversampling methods usually result in more serious class overlapping due to improper choice of the reference samples. To circumvent this shortcoming, according to the different possibilities of minority class samples appearing in the overlapping regions in the feature space, a grouping scheme for the minority class samples is first designed to identify the overlapping region samples. Then, a new oversampling method based on this grouping scheme is proposed to make the new samples far away from the overlapping region and rectify the decision boundary properly. Subsequently, a new effective classification algorithm is developed for imbalanced data. Extensive experiments show that the proposed algorithm is superior to the seventeen benchmark algorithms in terms of three performance metrics, especially on high imbalance ratio data sets.

A Weakly Supervised Learning-Based Oversampling Framework for Class-Imbalanced Fault Diagnosis

With the lack of failure data, class imbalance has become a common challenge in the fault diagnosis of industrial systems. The oversampling methods can tackle the class-imbalanced problem by generating the minority samples to balance the training set. However, one of the main challenges of the existing oversampling methods is how to generate high-quality minority samples. Traditional oversampling methods regard all synthetic samples as minority ones to be added to the training set without filtering. The low-quality synthetic samples would distort the distribution of the dataset and worsen the classification performance. In this article, we propose a weakly supervised oversampling method that treats all synthetic samples as unlabeled samples and develops a graph semisupervised learning algorithm to select high-quality synthetic samples, adding into the final training set as minority samples. To improve the quality of synthetic samples, we propose a cost-sensitive neighborhood component analysis dimensionality reduction method to enhance domain information validity in high-dimensional datasets. Finally, combining a boosting-based ensemble framework, we propose a new imbalanced learning framework suitable for high dimensionality and highly imbalanced fault diagnosis in industrial systems. The experimental validation is performed on five real-world wind turbine blade cracking failure datasets and compared to 15 benchmark methods. The experimental results show that average performances and robustness of the proposed framework are significantly better than those of the benchmark methods.

Selective oversampling approach for strongly imbalanced data.

Challenges posed by imbalanced data are encountered in many real-world applications. One of the possible approaches to improve the classifier performance on imbalanced data is oversampling. In this paper, we propose the new selective oversampling approach (SOA) that first isolates the most representative samples from minority classes by using an outlier detection technique and then utilizes these samples for synthetic oversampling. We show that the proposed approach improves the performance of two state-of-the-art oversampling methods, namely, the synthetic minority oversampling technique and adaptive synthetic sampling. The prediction performance is evaluated on four synthetic datasets and four real-world datasets, and the proposed SOA methods always achieved the same or better performance than other considered existing oversampling methods.

Radius-SMOTE: A New Oversampling Technique of Minority Samples Based on Radius Distance for Learning From Imbalanced Data

Imbalanced learning problems are a challenge faced by classifiers when data samples have an unbalanced distribution in each class. Furthermore, the synthetic oversampling method (SMOTE) is a preprocessing technique widely used to synthesize new data and balance the different numbers of samples in each class. One of the SMOTE method’s expansions is based on the initial selection approach, which determines the best candidates to be oversampled in the data before the process of synthetic example generation starts. However, SMOTE and most of the existing oversampling methods based on initial selection still found overlapping data on the final result. This issue makes it difficult for any classifiers to determine the decision boundary of each class. Therefore, this research proposes a new oversampling technique called Radius-SMOTE, which emphasizes the initial selection approach by creating synthetic data based on a safe radius distance. Furthermore, new synthetic data are prevented from overlapping in the opposite class with the safe radius distance. The Radius-SMOTE was evaluated extensively with thirteen artificial imbalanced datasets from the KEEL repository. The experimental results show that the proposed method is able to achieve the best results on 5 datasets, namely yeast-1-4-5-8_vs_7, ecoli-0-1-3-7_vs_2-6, Umbilical cord, Pima, and Haberman dataset in term of various assessment metrics. Besides that, the computational cost for our proposed method is also relatively low, with an average time of 0.5 to 1 second on the 13 tested datasets.

Constrained Oversampling: An Oversampling Approach to Reduce Noise Generation in Imbalanced Datasets With Class Overlapping

Imbalanced datasets are pervasive in classification tasks and would cause degradation of the performance of classifiers in predicting minority samples. Oversampling is effective in resolving the class imbalance problem. However, existing oversampling methods generally introduce noise examples into original datasets, especially when the datasets contain class overlapping regions. In this study, a new oversampling method named Constrained Oversampling is proposed to reduce noise generation in oversampling. This algorithm first extracts overlapping regions in the dataset. Then Ant Colony Optimization is applied to define the boundaries of minority regions. Third, oversampling under constraints is employed to synthesize new samples to get a balanced dataset. Our proposal distinguishes itself from other techniques by incorporating constraints in the oversampling process to inhibit noise generation. Experiments show that it outperforms various benchmark oversampling approaches. The explanation for the effectiveness of our method is given by studying the impact of class overlapping on imbalanced learning.

A Synthetic Minority Based on Probabilistic Distribution (SyMProD) Oversampling for Imbalanced Datasets

Handling an imbalanced class problem is a challenging task in real-world applications. This problem affects various prediction models that predict only the majority class and fail to identify the minority class because of the skewed data. The oversampling technique is one of the exciting solutions that handles the imbalanced class problem. However, several existing oversampling methods do not consider the distribution of the target variable and cause an overlapping class problem. Therefore, this study introduces a new oversampling technique, namely Synthetic Minority based on Probabilistic Distribution (SyMProD), to handle skewed datasets. Our technique normalizes data using a Z-score and removes noisy data. Then, the proposed method selects minority samples based on the probability distribution of both classes. The synthetic instances are generated from selected points and several minority nearest neighbors. Our technique aims to create synthetic instances that cover the minority class distribution, avoid the noise generation, and reduce the possibilities of overlapping classes and overgeneralization problems. Our proposed technique is validated using 14 benchmark datasets and three classifiers. Moreover, we compare the performance with seven other conventional oversampling algorithms. The empirical results show that our method achieves better performance compared with other oversampling techniques.

A Local Adaptive Minority Selection and Oversampling Method for Class-Imbalanced Fault Diagnostics in Industrial Systems

Data-driven fault diagnostics of industrial systems suffer from class-imbalanced problems, which is a common challenge for machine learning algorithms as it is difficult to learn the features of the minority class samples. Synthetic oversampling methods are commonly used to tackle these problems by generating minority class samples to balance the majority and minority classes. Two major issues will influence the performance of oversampling methods which are how to choose the most appropriate existing minority seed samples, and how to synthesize new samples from seed samples effectively. However, many existing oversampling methods are not accurate and effective enough to generate new samples when dealing with high-dimensional faulty samples with different imbalanced ratios, since they do not take these two factors into consideration at the same time. This article develops a novel adaptive oversampling technique: expectation maximization (EM)-based local-weighted minority oversampling technique for industrial fault diagnostics. This method uses a local-weighted minority oversampling strategy to identify hard-to-learn informative minority fault samples and an EM-based imputation algorithm to generate fault samples based on the distribution of minority samples. To validate the performance of the developed method, experiments were conducted on two real-world datasets. The results show that the developed method can achieve better performances, in terms of F-measure, Matthews correlation coefficient (MCC), and Mean (average of F-measure and MCC) values, on multiclass imbalanced fault diagnostics in different imbalance ratios than state-of-arts' baseline sampling techniques.

Real-value negative selection over-sampling for imbalanced data set learning

The learning problem from imbalanced data set poses a major challenge in data mining community. Conventional machine learning algorithms show poor performance in dealing with the classification problems of imbalanced data set since they are originally designed to work with balanced class distribution. In this paper, we propose a new over-sampling technique, which uses the real-value negative selection (RNS) procedure to generate artificial minority data with no requirement of actual minority data available. The generated minority data with rare actual minority data if available are combined with the majority data as input to a bi-class classification approach for learning. In the experiments, we demonstrate the effectiveness of RNS in avoiding the problems often encountered by the existing over-sampling methods such as the generation of noisy instances and almost duplicated instances in the same clusters. Moreover, the extensive experimental results on the different imbalanced datasets from UCI repository and real-world imbalanced datasets show that when dealing with the classification of imbalanced datasets, the proposed hybrid approach can achieve better performance in terms of both G-Mean and F-Measure evaluation metrics as compared to the other existing imbalanced dataset classification techniques.

A Parameter-Free Cleaning Method for SMOTE in Imbalanced Classification

Oversampling is an efficient technique in dealing with class-imbalance problem. It addresses the problem by reduplicating or generating the minority class samples to balance the distribution between the samples of the majority and the minority class. Synthetic minority oversampling technique (SMOTE) is one of the typical representatives. During the past decade, researchers have proposed many variants of SMOTE. However, the existing oversampling methods may generate wrong minority class samples in some scenarios. Furthermore, how to effectively mine the inherent complex characteristics of imbalanced data remains a challenge. To this end, this paper proposes a parameter-free data cleaning method to improve SMOTE based on constructive covering algorithm. The dataset generated by SMOTE is first partitioned into a group of covers, then the hard-to-learn samples can be detected based on the characteristics of sample space distribution. Finally, a pair-wise deletion strategy is proposed to remove the hard-to-learn samples. The experimental results on 25 imbalanced datasets show that our proposed method is superior to the comparison methods in terms of various metrics, such as ${F}$ -measure, ${G}$ -mean, and Recall. Our method not only can reduce the complexity of the dataset but also can improve the performance of the classification model.

SMS Phishing Detection Using Oversampling and Feature Optimization Method

In this paper, a new SMS phishing detection method using oversampling and feature optimization technology is proposed to improve SMS phishing detection accuracy. Three types features are presented including token features, topic features and Linguistic Inquiry and Word Count (LIWC) features. One of the existing oversampling methods called Adaptive Synthetic Sampling Approach is applied in this paper since it has good performance. Then, Binary Particle Swarm Optimization(BPSO) algorithm is used to analyze the three types features and select the optimal combination of all the features. Finally, the detection results are achieved by Random Forest classification algorithm. Experimental results show oversampling method and feature optimization method improve the accuracy of SMS phishing detection. The best accuracy of the proposed method is 99.01% with an average of 86.6 features. The results demonstrate that the proposed method has a promising performance for SMS phishing detection.

Fuzzy rule-based oversampling technique for imbalanced and incomplete data learning

Datasets that have skewed class distributions pose a difficulty to learning algorithms in pattern classification. A number of different methods to deal with this problem have been developed in recent years. Specifically, synthetic oversampling techniques focus on balancing the distribution between the training instances of the majority and minority classes by generating extra artificial minority class instances. Unfortunately, few of them can be spread to tackle the problem of imbalanced data with missing values. Moreover, in most cases, existing oversampling methods do not make full use of the correlation between attributes. To this end, in this paper, we propose a fuzzy rule-based oversampling technique (FRO) to handle the class imbalance problem. FRO firstly creates fuzzy rules from the training data and assigns each of them a rule weight, which represents the certainty degree of an instance belonging to the fuzzy subspace. Then it synthesizes new minority instances under the guidance of fuzzy rules. The number of minority instances to be generated under a given fuzzy rule is determined by the rule weight. In a similar way, FRO can also recover the missing values that exist in the imbalanced dataset. Extensive experiments using 55 real-world imbalanced datasets evaluate the performance of the proposed FRO technique. The results show that our method is better than or comparable with a set of alternative state-of-the-art imbalanced classification algorithms in terms of various assessment metrics.

A new segmented oversampling method for imbalanced data classification using quasi‐linear SVM

Data imbalance occurs on most real‐world classification problems and decreases the performance of classifiers. An oversampling method addresses the imbalance problem by generating synthetic samples to balance the data distribution. However, many of the existing oversampling methods have potential problems in generating wrong and unnecessary synthetic samples, which makes the learning tasks difficult. This paper proposes a new segmented oversampling method for imbalanced data classification. The input space is first partitioned into several linearly separable local partitions along the potential separation boundary by introducing a bottom‐up, minimal‐spanning‐tree‐based clustering method; an oversampling method is then applied within each local linear partition to prevent the generation of wrong and unnecessary synthetic samples; a quasi‐linear support vector machine is finally used to realize the classification by taking advantages of the local linear partitions. Simulation results on different real‐world datasets show that the proposed segmented oversampling method is effective for imbalanced data classifications. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Cluster-based Weighted Oversampling for Ordinal Regression (CWOS-Ord)

A new oversampling method called Cluster-based Weighted Oversampling for Ordinal Regression (CWOS-Ord) is proposed for addressing ordinal regression with imbalanced datasets. Ordinal regression is a supervised approach for learning the ordinal relationship between classes. In many applications, the dataset is highly imbalanced where the instances of some classes (majority classes) occur much more frequently than instances of other classes (minority classes). This significantly degrades the classification performance as classifiers tend to strongly favor the majority classes. Standard oversampling methods can be used to improve the dataset class distribution; however, they do not consider the ordinal relationship between the classes. The proposed CWOS-Ord method aims to address this problem by first clustering minority classes and then oversampling them based on their distances and ordering relationship to other classes’ instances. The final size to oversample the clusters depends on their complexity and their initial size so that more synthetic instances are generated for more complex and smaller clusters while fewer instances are generated for less complex and larger clusters. As a secondary contribution, existing oversampling methods for two-class classification have been extended for ordinal regression. Results demonstrate that the proposed CWOS-Ord method provides significantly better results compared to other methods based on the performance measures.

MWMOTE--Majority Weighted Minority Oversampling Technique for Imbalanced Data Set Learning

Imbalanced learning problems contain an unequal distribution of data samples among different classes and pose a challenge to any classifier as it becomes hard to learn the minority class samples. Synthetic oversampling methods address this problem by generating the synthetic minority class samples to balance the distribution between the samples of the majority and minority classes. This paper identifies that most of the existing oversampling methods may generate the wrong synthetic minority samples in some scenarios and make learning tasks harder. To this end, a new method, called Majority Weighted Minority Oversampling TEchnique (MWMOTE), is presented for efficiently handling imbalanced learning problems. MWMOTE first identifies the hard-to-learn informative minority class samples and assigns them weights according to their euclidean distance from the nearest majority class samples. It then generates the synthetic samples from the weighted informative minority class samples using a clustering approach. This is done in such a way that all the generated samples lie inside some minority class cluster. MWMOTE has been evaluated extensively on four artificial and 20 real-world data sets. The simulation results show that our method is better than or comparable with some other existing methods in terms of various assessment metrics, such as geometric mean (G-mean) and area under the receiver operating curve (ROC), usually known as area under curve (AUC).

LVQ-SMOTE – Learning Vector Quantization based Synthetic Minority Over–sampling Technique for biomedical data

BackgroundOver-sampling methods based on Synthetic Minority Over-sampling Technique (SMOTE) have been proposed for classification problems of imbalanced biomedical data. However, the existing over-sampling methods achieve slightly better or sometimes worse result than the simplest SMOTE. In order to improve the effectiveness of SMOTE, this paper presents a novel over-sampling method using codebooks obtained by the learning vector quantization. In general, even when an existing SMOTE applied to a biomedical dataset, its empty feature space is still so huge that most classification algorithms would not perform well on estimating borderlines between classes. To tackle this problem, our over-sampling method generates synthetic samples which occupy more feature space than the other SMOTE algorithms. Briefly saying, our over-sampling method enables to generate useful synthetic samples by referring to actual samples taken from real-world datasets.ResultsExperiments on eight real-world imbalanced datasets demonstrate that our proposed over-sampling method performs better than the simplest SMOTE on four of five standard classification algorithms. Moreover, it is seen that the performance of our method increases if the latest SMOTE called MWMOTE is used in our algorithm. Experiments on datasets for β-turn types prediction show some important patterns that have not been seen in previous analyses.ConclusionsThe proposed over-sampling method generates useful synthetic samples for the classification of imbalanced biomedical data. Besides, the proposed over-sampling method is basically compatible with basic classification algorithms and the existing over-sampling methods.