Synthetic Minority Class Samples Research Articles

Oversampling framework based on sample subspace optimization with accelerated binary particle swarm optimization for imbalanced classification

In response to the need to generate synthetic minority class samples to extend minority classes, the SMOTE-based oversampling methods have been favored for class-imbalanced classification. They usually generate unnecessary noise when training data are not well separated. Although filtering-based oversampling methods are recognized as effective solutions for addressing noise generation through employing specific noise filters based on instance selection methods to remove suspicious noise, they suffer from the following issues: a) noise filters heavily rely on strong assumptions, causing low robustness to different datasets; b) noise filters are specially designed for a specific oversampling method, and are not easily extended to others; and c) noise filters have a relatively high time consumption. To address noise generation while overcoming the above issues a)-c), an oversampling framework based on sample subspace optimization with accelerated binary particle swarm optimization (OF-SSO-ABPSO) is proposed. OF-SSO-ABPSO is a wrapping framework compatible with almost all the oversampling methods. First, in the framework, a SMOTE-based method is used to generate synthetic minority class samples. Second, a novel accelerated binary particle swarm optimization (ABPSO) algorithm with a new search space reduction strategy, a new particle update mechanism, and a new fitness function is proposed. Third, a novel ABPSO-based sample subspace optimization (SSO-ABPSO) method is proposed and used as a noise filter to remove suspicious noise from the training set and synthetic minority class samples. Experiments prove that, a) OF-SSO-ABPSO can improve 6 representative SMOTE variations by addressing noise generation, and b) OF-SSO-ABPSO outperforms 7 state-of-the-art filtering-based oversampling methods.

An overlapping minimization-based over-sampling algorithm for binary imbalanced classification

Imbalanced learning is an important branch of machine learning. It addresses the challenge of improving classifier accuracy for minority classes in imbalanced data sets. Currently, the mainstream methods for handling imbalanced learning are the synthetic minority oversampling technique (SMOTE) and its variants, which generate synthetic minority class samples to balance the dataset. However, existing methods suffer from issues such as increased sample overlap, exacerbated intra-class imbalance, and are sensitive to parameter settings. These issues make it challenging to generate high-quality minority class samples and can adversely affect the dataset. To address these challenges, this study proposes a novel overlapping minimization-based over-sampling (OMOS) algorithm for binary imbalanced classification. The OMOS algorithm consists of four steps: clustering, filtering, auto-encoding, and oversampling. In the clustering step, the mean shift algorithm is utilized to cluster the original dataset and identify clusters that belong to the minority class. In the filtering step, safe samples are selected that maintain consistent labels before and after clustering. Then, in the auto-encoding step, autoencoders are utilized to capture the distribution characteristics of safe samples within each minority class cluster. Finally, in the last step, minority class samples are generated based on the probability distribution learned from safe samples. Furthermore, OMOS introduces a novel approach to compute suitable sampling rates for each minority class cluster to handle intra-class imbalance. Experimental results show that the proposed OMOS algorithm outperforms six state-of-the-art SMOTE-based oversampling algorithms on 20 real-world imbalanced datasets and four classifiers: naive Bayes classifier, support vector machine, logistic regression, and decision trees. This demonstrates that OMOS is effective for binary imbalanced classification tasks.

An explainable ensemble approach for advanced brain tumor classification applying Dual-GAN mechanism and feature extraction techniques over highly imbalanced data.

Brain tumors are one of the leading diseases imposing a huge morbidity rate across the world every year. Classifying brain tumors accurately plays a crucial role in clinical diagnosis and improves the overall healthcare process. ML techniques have shown promise in accurately classifying brain tumors based on medical imaging data such as MRI scans. These techniques aid in detecting and planning treatment early, improving patient outcomes. However, medical image datasets are frequently affected by a significant class imbalance, especially when benign tumors outnumber malignant tumors in number. This study presents an explainable ensemble-based pipeline for brain tumor classification that integrates a Dual-GAN mechanism with feature extraction techniques, specifically designed for highly imbalanced data. This Dual-GAN mechanism facilitates the generation of synthetic minority class samples, addressing the class imbalance issue without compromising the original quality of the data. Additionally, the integration of different feature extraction methods facilitates capturing precise and informative features. This study proposes a novel deep ensemble feature extraction (DeepEFE) framework that surpasses other benchmark ML and deep learning models with an accuracy of 98.15%. This study focuses on achieving high classification accuracy while prioritizing stable performance. By incorporating Grad-CAM, it enhances the transparency and interpretability of the overall classification process. This research identifies the most relevant and contributing parts of the input images toward accurate outcomes enhancing the reliability of the proposed pipeline. The significantly improved Precision, Sensitivity and F1-Score demonstrate the effectiveness of the proposed mechanism in handling class imbalance and improving the overall accuracy. Furthermore, the integration of explainability enhances the transparency of the classification process to establish a reliable model for brain tumor classification, encouraging their adoption in clinical practice promoting trust in decision-making processes.

An Integrated Feature Extraction Based on Principal Components and Deep Auto Encoder with Extra Tree for Intrusion Detection Systems

With advances in computer network technology, the Internet has become an integral part of our daily lives. It goes without saying that providing security to network data is crucial. To this effect, the intrusion detection system (IDS) is vital for defending networks against cyberattacks. However, the high dimensionality of data is a significant issue that impacts categorisation accuracy. Therefore, we proposed a novel integrated feature extraction method called PC-UDAE that uses principal component analysis (PCA) and Unsupervised Deep Autoencoder (UDAE) to extract linear and nonlinear relationships. Also, to address the class imbalance, synthetic minority class samples are generated using the combination of Synthetic Minority Over Sampling Technique and Edited Nearest Neighbour (SMOTE-ENN). Finally, the extracted features are trained by using supervised machine learning models like Random Forest (RF), Extreme Gradient Boosting Machine (XGBM), Decision Tree (DT), Light Gradient Boosting Machine (LGBM), Extra Tree (ET), Support Vector Machine (SVM), AdaBoost (AB), and K-Nearest Neighbour (KNN) with the original imbalanced and balanced data. We analysed our suggested model using UNSW-NB 15, NSL-KDD, Ton-IoT data sets and obtained 98.85%, 99.59%, and 99.97% accuracy, respectively. Our experimental findings show that our proposed method outperformed all other competing methods.

CTCN: a novel credit card fraud detection method based on Conditional Tabular Generative Adversarial Networks and Temporal Convolutional Network.

Credit card fraud can lead to significant financial losses for both individuals and financial institutions. In this article, we propose a novel method called CTCN, which uses Conditional Tabular Generative Adversarial Networks (CTGAN) and temporal convolutional network (TCN) for credit card fraud detection. Our approach includes an oversampling algorithm that uses CTGAN to balance the dataset, and Neighborhood Cleaning Rule (NCL) to filter out majority class samples that overlap with the minority class. We generate synthetic minority class samples that conform to the original data distribution, resulting in a balanced dataset. We then employ TCN to analyze transaction sequences and capture long-term dependencies between data, revealing potential relationships between transaction sequences, thus achieving accurate credit card fraud detection. Experiments on three public datasets demonstrate that our proposed method outperforms current machine learning and deep learning methods, as measured by recall, F1-Score, and AUC-ROC.

Synthetic Minority Oversampling Technique Based on Adaptive Noise Optimization and Fast Search for Local Sets for Random Forest

The classification is usually degraded due to the imbalanced class distribution. Synthetic minority oversampling technique (SMOTE) has been successful in improving imbalanced classification and has received great praise. Overgeneralization is one of the most challenges in SMOTE. Although multiple SMOTE-based variations are proposed against overgeneralization, they still have the following shortcomings: (a) creating too many synthetic samples in high-density regions; (b) removing suspicious noise directly instead of modifying them; (c) relying on many parameters. This paper proposes a new SMOTE based on adaptive noise optimization and fast search for local sets (SMOTE-ANO-FLS) to overcome the overgeneralization and the shortcomings of existing works. First, SMOTE-ANO-FLS uses the [Formula: see text]-D tree to fast search the local sets for each sample. Second, a new noise detection method based on local sets and the imbalanced ratio is proposed to detect suspicious noise. Third, a new adaptive noise optimization method is proposed to modify detected suspicious noise instead of removing them. Finally, a new probability weight based on local sets is proposed to help create more synthetic minority class samples in borderline and sparse regions. The effectiveness of SMOTE-ANO-FLS is proven by employing 7 oversampling methods and random forest on the extensive synthetic and real data sets.

SMOTE-LMKNN: A Synthetic Minority Oversampling Technique Based on Local Means-Based k-Nearest Neighbor

Traditional classifiers are trapped by the class-imbalanced problem due to the fact that they are biased toward the majority class. Oversampling methods can improve imbalanced classification by creating synthetic minority class samples. Noise generation has been a great challenge in oversampling methods. Filtering-based and direction-change methods are proposed against noise generation. Yet, the adopted noise filters in filtering-based methods are biased to the majority class. Besides, the [Formula: see text]-nearest neighbor (KNN)-based interpolation in filtering-based and direction-change methods is susceptible to abnormal samples (e.g. outliers, noise or unsafe borderline samples). To overcome noise generation while solving the above shortcomings of filtering-based and direction-change methods, this work presents a new synthetic minority oversampling technique based on local means-based KNN (SMOTE-LMKNN). In SMOTE-LMKNN, the local mean-based KNN (LMKNN) is first introduced to describe the local characteristic of imbalanced data. Second, a new LMKNN-based noise filter is proposed to remove noise and unsafe borderline samples. Third, the interpolation between a base sample and its LMKNN is proposed to create synthetic minority class samples. Empirical results of extensive experiments with 18 data sets show that SMOTE-LMKNN is competent compared with seven popular oversampling methods in training KNN classifier and classification and regression tree (CART).

Synthetic Minority Oversampling Technique Based on Adaptive Local Mean Vectors and Improved Differential Evolution

SMOTE is a classical oversampling method and aims to improve imbalanced classification by creating synthetic minority class samples. Overgeneralization is a great challenge in SMOTE and its improvements. Multiple variations of SMOTE are proposed against imbalances between classes and overgeneralization. However, they still have the following issues: a) most methods depend on too many parameters; b) most methods fail to detect suspicious noise effectively and modify them; c) interpolation of almost all methods is susceptible to abnormal samples. To overcome the above issues, a new synthetic minority oversampling technique based on adaptive local mean vectors and improved differential evolution (SMOTE-LMVDE) is proposed. First, a new noise detection technique based on the defined adaptive local mean vectors (NDALMV) is proposed to find suspicious noise. Second, an improved differential evolution is proposed to modify and improve detected suspicious noise. Finally, a new interpolation based on the defined adaptive local mean vectors is proposed to create synthetic minority class samples. Experiments prove that the proposed method superior to 7 popular oversampling approaches on extensive data sets in the training nearest neighbor classifier and the decision tree classifier.

Genetic algorithm-based oversampling approach to prune the class imbalance issue in software defect prediction

Class imbalance is the potential problem that has been existent in machine learning, which hinders the performance of the classification algorithm when applied in real-world applications such as electricity pilferage, fraudulent transactions, anomaly detection, and prediction of rare diseases. Class imbalance refers to the problem where the distribution of the sample is skewed or biased toward one particular class. Due to its intrinsic nature the software fault prediction dataset falls into the same category where the software modules contain fewer defective modules compared to the non-defective modules. The majority of the oversampling techniques that has been proposed is to address the issue by generating synthetic samples of minority class to balance the dataset. But the synthetic samples generated are near duplicates that also results in over-generalization issue. We thus propose a novel oversampling approach to introduce synthetic samples using genetic algorithm (GA). GA is a form of evolutionary algorithm that employs biologically inspired techniques such as inheritance, mutation, selection, and crossover. The proposed algorithm generates synthetic sample of minority class based on the distribution measure and ensures that the samples are diverse within the class and are efficient. The proposed oversampling algorithm has been compared with SMOTE, BSMOTE, ADASYN, random oversampling, MAHAKIL, and no sampling approach with 20 defect prediction datasets from the promise repository and five prediction models. The results indicate that the genetic algorithm oversampling approach improves the fault prediction performance and reduced false alarm rate.

Creating synthetic minority class samples based on autoencoder extreme learning machine

This paper reports a new method (simplified as AE-ELM-SynMin) to create the Synthetic Minority class samples for imbalanced classification based on AutoEncoder Extreme Learning Machine (AE-ELM). AE-ELM-SynMin first trains an AE-ELM which is a special ELM with the same input and output, i.e., the original minority class samples. Second, the crossover, mutation and filtration operations are conducted on the hidden-layer output of AE-ELM and then the synthetic hidden-layer output is obtained. Third, the synthetic minority class samples are created by decoding the synthetic hidden-layer output with output-layer weights of AE-ELM. AE-ELM-SynMin guarantees that the synthetic minority class has the higher information amount than original minority class and meanwhile keeps the consistent probability distribution with the original minority class. The experimental results demonstrate the better imbalanced classification performances of AE-ELM-SynMin in comparison with the regular synthetic minority over-sampling technique (Regular-SMOTE) and its variants, e.g., Borderline-SMOTE, Random-SMOTE, and SMOTE-IPF.

Adversarial Semi-Supervised Learning for Diagnosing Faults and Attacks in Power Grids

This paper proposes a novel adversarial scheme for learning from data under harsh learning conditions of partially labelled samples and skewed class distributions. This novel scheme integrates the generative ability of the state-of-the-art conditional generative adversarial network with the semi-supervised deep ladder network and semi-supervised deep auto-encoder. The proposed generative-adversarial based semi-supervised learning framework, named GBSS, is a triple network that aims to optimize a newly defined objective function to enhance the performance of the semi-supervised learner with the help of a generator and discriminator. The duel between the generator and discriminator results in the generation of more synthetic minority class samples that are very similar to the original minority samples (attacks and faults). Meanwhile, GBSS trains the semi-supervised model to learn the general distribution of the minority class samples including the newly generated samples in contrast to other classes and iteratively adjusts its weights. Moreover, a diagnostic framework is designed, in which GBSS and several state-of-the-art semi-supervised learners are used for learning and diagnosing attacks and faults in power grids. These methods are evaluated and compared for diagnosing attacks and faults in two different power grid cases. The attained results demonstrate the superiority of GBSS in diagnosing attacks and faults under the harsh conditions.

Data augmentation-based conditional Wasserstein generative adversarial network-gradient penalty for XSS attack detection system.

The rapid growth of the worldwide web and accompanied opportunities of web applications in various aspects of life have attracted the attention of organizations, governments, and individuals. Consequently, web applications have increasingly become the target of cyberattacks. Notably, cross-site scripting (XSS) attacks on web applications are increasing and have become the critical focus of information security experts’ reports. Machine learning (ML) technique has significantly advanced and shown impressive results in the area of cybersecurity. However, XSS training datasets are often limited and significantly unbalanced, which does not meet well-developed ML algorithms’ requirements and potentially limits the detection system efficiency. Furthermore, XSS attacks have multiple payload vectors that execute in different ways, resulting in many real threats passing through the detection system undetected. In this study, we propose a conditional Wasserstein generative adversarial network with a gradient penalty to enhance the XSS detection system in a low-resource data environment. The proposed method integrates a conditional generative adversarial network and Wasserstein generative adversarial network with a gradient penalty to obtain necessary data from directivity, which improves the strength of the security system over unbalance data. The proposed method generates synthetic samples of minority class that have identical distribution as real XSS attack scenarios. The augmented data were used to train a new boosting model and subsequently evaluated the model using a real test dataset. Experiments on two unbalanced XSS attack datasets demonstrate that the proposed model generates valid and reliable samples. Furthermore, the samples were indistinguishable from real XSS data and significantly enhanced the detection of XSS attacks compared with state-of-the-art methods.

An improved ELM-based and data preprocessing integrated approach for phishing detection considering comprehensive features

In this paper, a novel approach based on non-inverse matrix online sequence extreme learning machine (NIOSELM) for phishing detection is presented, which takes into account three types of features to comprehensively characterize a website. For the NIOSELM algorithm, we use Sherman Morriso Woodbury equation to avoid the matrix inversion operation, and introduce the idea of online sequence extreme learning machine (OSELM) to update the training model. In order to reduce the dependence of the detection model on the majority class, we use Adaptive Synthetic Sampling (ADASYN) algorithm to generate the synthetic minority class samples to balance the distribution between the samples of the majority and minority classes. Furthermore, an improved denoising auto-encoder (SDAE) is designed to reduce the dimension of the experimental dataset. The experimental results show the efficiency and feasibility of the proposed detection mechanism. Moreover, the overall detection performance of NIOSELM is better than that of other existing methods, especially in training speed and the detection accuracy.

Transfer synthetic over-sampling for class-imbalance learning with limited minority class data

The problem of limited minority class data is encountered in many class imbalanced applications, but has received little attention. Synthetic over-sampling, as popular class-imbalance learning methods, could introduce much noise when minority class has limited data since the synthetic samples are not i.i.d. samples of minority class. Most sophisticated synthetic sampling methods tackle this problem by denoising or generating samples more consistent with ground-truth data distribution. But their assumptions about true noise or ground-truth data distribution may not hold. To adapt synthetic sampling to the problem of limited minority class data, the proposed Traso framework treats synthetic minority class samples as an additional data source, and exploits transfer learning to transfer knowledge from them to minority class. As an implementation, TrasoBoost method firstly generates synthetic samples to balance class sizes. Then in each boosting iteration, the weights of synthetic samples and original data decrease and increase respectively when being misclassified, and remain unchanged otherwise. The misclassified synthetic samples are potential noise, and thus have smaller influence in the following iterations. Besides, the weights of minority class instances have greater change than those of majority class instances to be more influential. And only original data are used to estimate error rate to be immune from noise. Finally, since the synthetic samples are highly related to minority class, all of the weak learners are aggregated for prediction. Experimental results show TrasoBoost outperforms many popular class-imbalance learning methods.

Imputation-based Ensemble Techniques for Class Imbalance Learning

Correct classification of rare samples is a vital data mining task and of paramount importance in many research domains. This article mainly focuses on the development of the novel class-imbalance learning techniques, which make use of oversampling methods integrated with bagging and boosting ensembles. Two novel oversampling strategies based on the single and the multiple imputation methods are proposed. The proposed techniques aim to create useful synthetic minority class samples, similar to the original minority class samples, by estimation of missing values that are already induced in the minority class samples. The re-balanced datasets are then used to train base-learners of the ensemble algorithms. In addition, the proposed techniques are compared with the commonly used class imbalance learning methods in terms of three performance metrics including AUC, F-measure, and G-mean over several synthetic binary class datasets. The empirical results show that the proposed multiple imputation-based oversampling combined with bagging significantly outperforms other competitors.

MWMOTE--Majority Weighted Minority Oversampling Technique for Imbalanced Data Set Learning

Imbalanced learning problems contain an unequal distribution of data samples among different classes and pose a challenge to any classifier as it becomes hard to learn the minority class samples. Synthetic oversampling methods address this problem by generating the synthetic minority class samples to balance the distribution between the samples of the majority and minority classes. This paper identifies that most of the existing oversampling methods may generate the wrong synthetic minority samples in some scenarios and make learning tasks harder. To this end, a new method, called Majority Weighted Minority Oversampling TEchnique (MWMOTE), is presented for efficiently handling imbalanced learning problems. MWMOTE first identifies the hard-to-learn informative minority class samples and assigns them weights according to their euclidean distance from the nearest majority class samples. It then generates the synthetic samples from the weighted informative minority class samples using a clustering approach. This is done in such a way that all the generated samples lie inside some minority class cluster. MWMOTE has been evaluated extensively on four artificial and 20 real-world data sets. The simulation results show that our method is better than or comparable with some other existing methods in terms of various assessment metrics, such as geometric mean (G-mean) and area under the receiver operating curve (ROC), usually known as area under curve (AUC).