Within-class Imbalance Research Articles

A MeanShift-guided oversampling with self-adaptive sizes for imbalanced data classification

The imbalanced data classification has gained popularity in machine learning research domain due to its prevalence in numerous applications and its difficulty. However, the majority of contemporary work primarily focuses on addressing between-class imbalance issues. Previous researches have shown that combined with other elements, such as within-class imbalance, small sample size and the presence of small disjuncts, the imbalanced data significantly increase the difficulties for the traditional classifiers to learn. Therefore, we propose a novel MeanShift-guided oversampling with self-adaptive sizes for imbalanced data classification. The proposed MeanShift-guided oversampling technique can simultaneously consider the distribution of minority class and majority class within the sphere with the current minority instance as its center, which can favor addressing small sample size and avoiding overlapping issues often caused by the nearest neighbor (NN)-based oversampling techniques. The incorporation of random vector and flexible cut-off mechanism for vector length can enhance the diversity among the generated synthetic minority instances and avoid overlapping, which makes it suitable for small sample size and small disjuncts problems. To address between-class and within-class imbalance issues, we also introduce a self-adaptive sizes assignment strategy for each minority instance to be oversampled, where the assigned size is inversely proportional to its density and its distance from the majority class. In addition to eliminating within-class imbalance, the strategy can ensure that the informative border minority instances have more opportunities to be oversampled, thus improving classification performance. Extensive experimental results on some datasets with different distributions and imbalance ratios show the proposed algorithm outperforms other compared ones with significant difference.

Review of resampling techniques for the treatment of imbalanced industrial data classification in equipment condition monitoring

In an actual industrial scenario, machines typically operate normally for the majority of the time, with malfunctions occurring only occasionally. As a result, there is very little recorded data on defects. Consequently, the fault diagnostic dataset becomes imbalanced, with a significantly lower number of fault samples compared to normal samples. Furthermore, with the rapid development of the manufacturing industry, the increasing complexity of machines and equipment leads to various challenges in collecting fault data, such as noise, within-class imbalance, multi-class imbalance, and time series imbalance. It is worth noting that this study is the first to comprehensively summarize these four specific challenges. Therefore, addressing these issues has become a critical research focus and a pain point in the field of fault diagnosis, and numerous solutions have emerged. This study provides a comprehensive overview of these solutions at three levels: data preprocessing, feature extraction, and classifier improvement. It also describes the applications of imbalanced data classification methods, including pure resampling techniques, as well as sampling techniques that combine resampling algorithms with feature extraction and classifier improvement in industrial scenarios. Finally, we summarize the challenges facing imbalanced data classification research and suggest potential directions for future studies.

Self-adaptive oversampling method based on the complexity of minority data in imbalanced datasets classification

Learning from imbalanced datasets is a nontrivial task for supervised learning community. Traditional classifiers may have difficulties to learn the concept related to the minority class when addressing imbalanced classification and the issues can become more deteriorated in the presence of other complicated aspects: overlapping, outliers and small disjuncts, etc. In this paper, we propose a self-adaptive oversampling algorithm based on the complexity of minority data for dealing with imbalanced datasets classification problems. In the proposed algorithm, various hyperspheres with different radii determined by imbalance ratio and the distances to the nearest enemy neighbors are firstly generated to cover all minority instances provided that they cannot contain any majority instance. Subsequently, the oversampling process is conducted only within these hyperspheres and thus the generated synthetic minority instances cannot intervene within the majority space, eventually avoiding overlapping issues during achieving between-class balance. In addition, a self-adaptive assignment strategy of oversampling sizes is developed based on the minority data complexity, where the hyperspheres with small radii and few instances in them are provided more chances to be oversampled. The strategy will favor addressing the outliers and small disjuncts issues since the hyperspheres covering the outliers and small disjuncts are usually of small sizes and contain few instances, which makes them have more chances to generate synthetic instances and thus eliminate within-class imbalance due to lack of density. Moreover, since the hyperspheres covering boundary minority instances are relatively small and thus are assigned with larger oversampling sizes, the proposed approach can also strengthen the boundary information of minority class, thus favoring the later learning tasks. The extensive experimental results on various simulated and real-world imbalanced datasets show that the proposed method significantly outperforms other state-of-the-art oversampling ones.

An imbalanced binary classification method via space mapping using normalizing flows with class discrepancy constraints

For imbalanced binary classification, most methods classify data in the original space by balancing data or modifying classification algorithms. However, for datasets with severe overlaps and complex class boundaries, the classification performance is limited by the ability of the classifier to fit the decision boundary. A practical alternative is to map the data to a latent space with lower classification difficulty. In this paper, we propose an imbalanced binary classification method via space mapping using normalizing flows with class discrepancy constraints. The flow-based model is employed to map the original data to a latent space, ensuring the accuracy of the mapping and the authenticity of the mapped distribution. To solve the problem of overlaps and within-class imbalance, class discrepancy constraints, including global and local constraints, are proposed to modify the flow-based model. The former maps different subclusters to the same cluster of the latent space, and the latter increases the separability of different classes. Classification thus can be carried out in the latent space with simpler distribution and high separability to achieve better classification performance. Experimental results on 35 KEEL and UCI public datasets indicate that the proposed method outperforms other state-of-the-art methods on F1-measure and G-mean.

Deep instance envelope network-based imbalance learning algorithm with multilayer fuzzy C-means clustering and minimum interlayer discrepancy

Imbalanced learning is important and challenging since the problem of the classification of imbalanced datasets is prevalent in machine learning and data mining fields. Sampling approaches are proposed to address this issue, and cluster-based oversampling methods have shown great potential as they aim to simultaneously tackle between-class and within-class imbalance issues. However, all existing clustering methods are based on a one-time approach. Due to the lack of a priori knowledge, improper setting of the number of clusters often exists, which leads to poor clustering performance. Besides, the existing methods are likely to generate noisy instances. To solve these problems, this paper proposes a deep instance envelope network-based imbalanced learning algorithm with the multilayer fuzzy c-means (MlFCM) and a minimum interlayer discrepancy mechanism based on the maximum mean discrepancy (MIDMD). This algorithm can guarantee high quality balanced instances using a deep instance envelope network in the absence of prior knowledge. First, the MlFCM is designed for the original minority class instances to obtain deep instances and increase the diversity of instances. Then, the MIDMD is proposed to avoid the generation of noisy instances and maintain the consistency of the interlayers of instances. Next, the multilayer FCM and minimum interlayer discrepancy mechanism are combined to construct a deep instance envelope network – the MlFC&IDMD. Finally, an imbalance learning algorithm is proposed based on the MlFC&IDMD. In the experimental section, thirty-three popular public datasets are used for verification, and over ten representative algorithms are used for comparison. The experimental results show that the proposed approach significantly outperforms other popular methods.

SVDD-based weighted oversampling technique for imbalanced and overlapped dataset learning

Imbalanced dataset classification issue poses a major challenge on machine learning domain. Traditional supervised learning algorithms usually bias towards the majority class when handling imbalanced datasets, thus leading to poor classification results on the minority class. The learning task would become crucially difficult when there are overlapping and within-imbalance issues in imbalanced datasets, which are often the case and have been proven to severely deteriorate the classification performance relative to between-class imbalance. In this paper, we propose a novel SVDD boundary-based weighted oversampling approach (SVDDWSMOTE) for handling imbalanced and overlapped data. The proposed approach first applies support vector data description (SVDD) model with greater penalty constant for the minority class than the majority class to generate the class boundary, and then identifies those misclassified majority or few minority instances by the generated class boundary as potential overlapped or noisy ones and eliminates them. To address the within-balance issues, we propose a weight assignment strategy based on densities and the distances to the SVDD class boundary, which facilitates simultaneously combating between-class and within-class imbalance issues caused by complicated distribution. In addition, such a strategy also favors generating more synthetic minority instances for borderline and sparser instances which are usually informative to the later learning tasks. Finally, oversampling is performed by the weighed SMOTE scheme based on SVDD boundary to not only counteract the within imbalance but also avoid the generation of any noisy or overlapped synthetic instance. Extensive comparison results on various datasets show that the proposed approach achieves statistically significant improvements in terms of different classification performance metrics relative to state-of-the-art ones.

SVDD boundary and DPC clustering technique-based oversampling approach for handling imbalanced and overlapped data

Imbalanced datasets classification remains an important domain in machine learning. Conventional supervised learning algorithms tend to be biased towards the majority class when addressing imbalanced datasets, thus providing poor classification results on the minority class. The learning task would become more challenging when there are overlapping and within-imbalance issues in imbalanced datasets, which are often the case and have been proven to play more negative impact on the classification performance than between-class imbalance. In this paper, we propose a novel SVDD (Support Vector Data Description) boundary and DPC (Density Peaks Clustering) clustering technique-based oversampling approach (SVDDDPCO) for handling imbalanced and overlapped data. The proposed approach first utilizes support vector data description (SVDD) model with greater penalty constant for the minority class than the majority class to generate the class boundary, and then identifies those misclassified majority or few minority instances by the boundary as potential overlapped or noisy ones and eliminates them. To address the within-balance issues, density peaks clustering (DPC) is used to cluster the minority instances due to its advantage of accurately identifying sub-clusters with different sizes and densities, which facilitates simultaneously combating between-class and within-class imbalance issues caused by various reasons. The minority instances are assigned with different weights inversely proportional to their local densities and their distances to the boundary and then the size for each identified sub-cluster to be oversampled is adaptively determined according to its own size and the weights of its elements. Such strategy aims to generate more synthetic minority instances for borderline and sparser small sub-clusters which are usually informative to the later learning tasks. Finally, oversampling is performed within each sub-cluster with different sizes to not only counteract the within imbalance but also avoid the generation of any noisy or overlapped synthetic instance. Extensive comparison results on various datasets showed the proposed method achieves statistically significant improvements in classification performance relative to state-of-the-art ones.

Improving Imbalanced Land Cover Classification with K-Means SMOTE: Detecting and Oversampling Distinctive Minority Spectral Signatures

Land cover maps are a critical tool to support informed policy development, planning, and resource management decisions. With significant upsides, the automatic production of Land Use/Land Cover maps has been a topic of interest for the remote sensing community for several years, but it is still fraught with technical challenges. One such challenge is the imbalanced nature of most remotely sensed data. The asymmetric class distribution impacts negatively the performance of classifiers and adds a new source of error to the production of these maps. In this paper, we address the imbalanced learning problem, by using K-means and the Synthetic Minority Oversampling Technique (SMOTE) as an improved oversampling algorithm. K-means SMOTE improves the quality of newly created artificial data by addressing both the between-class imbalance, as traditional oversamplers do, but also the within-class imbalance, avoiding the generation of noisy data while effectively overcoming data imbalance. The performance of K-means SMOTE is compared to three popular oversampling methods (Random Oversampling, SMOTE and Borderline-SMOTE) using seven remote sensing benchmark datasets, three classifiers (Logistic Regression, K-Nearest Neighbors and Random Forest Classifier) and three evaluation metrics using a five-fold cross-validation approach with three different initialization seeds. The statistical analysis of the results show that the proposed method consistently outperforms the remaining oversamplers producing higher quality land cover classifications. These results suggest that LULC data can benefit significantly from the use of more sophisticated oversamplers as spectral signatures for the same class can vary according to geographical distribution.

Global Data Distribution Weighted Synthetic Oversampling Technique for Imbalanced Learning

Imbalanced learning is a common problem in data mining. There is a different distribution of data samples among other classes in the imbalanced datasets. It's a challenge for standard algorithms designed for balanced class distributions. Although there are various strategies to solve this problem, generating artificial data to achieve a relatively balanced class distribution is universal rather than directly modifying specific classification algorithms. The oversampled data can be combined with any user-specified algorithm without any restrictions. In this paper, we present a novel oversampling method, Global Data Distribution Weighted Synthetic Oversampling Technique (GDDSYN). By applying clustering, optimizing the selection criteria of the minority class samples that are used to generate synthetic samples, avoiding generating more noise samples. GDDSYN assigns weights for the number of synthetic samples to tackle the within-class imbalance and between-class imbalance simultaneously, according to the informative level of the sample and the sparsity of the cluster to which the sample belongs. The use of scores with Silhouette Coefficient and Mutual Information helps the k-means algorithm set a reasonable number of clusters for the minority and majority classes respectively so that the clustering effect can be guaranteed. Next, by using clustering information, synthetic samples' generation path is improved to avoid class overlap. Additionally, GDDSYN has been evaluated extensively on 10 artificial and 10 real-world data sets. The empirical results show that our method is outperforms or comparable with some other existing methods in terms of assessment metrics when artificial data generated by GDDSYN are used.

A modified real-value negative selection detector-based oversampling approach for multiclass imbalance problems

A skewed distribution poses a major challenge in multiclass imbalanced problems and has attracted growing interest in the engineering and research communities. Conventional classifiers pose limitations in handling multiclass imbalanced data sets since they were originally designed to handle a balanced distribution. This paper proposes a modified real-value negative selection detector-based oversampling approach for the multiclass imbalance problem. Different from previous works, we have modified and introduced the traditional real-value negative selection algorithm to address the issue of the multiclass imbalance problem. First, the modified real-value negative selection technique is used to create detectors for each minority class. Then, a supervision mechanism is designed using these detectors to prevent overgeneralization. Furthermore, the method of selection weight based on crowding density and detectors is devised with the aim of reducing the within-class imbalance for each minority class. Finally, simulations were conducted on the public real-world multiclass imbalanced data sets from Knowledge Extraction based on Evolutionary Learning (KEEL) and the University of California Irvine (UCI). The experimental and statistical results have demonstrated that the proposed algorithm obtains better performance compared with seven well-known oversampling algorithms in terms of five metrics (precision, recall, F-measure, Multiclass G-mean (MG), and Multiclass Area Under the ROC (MAUC)) and four types of classifiers.

Detection of Radio Pulsars in Single-pulse Searches Within and Across Surveys

Pulsar detection using machine learning is a challenging problem as it involves extreme class imbalance and strong prioritization of high Recall. This paper is focused on automatic detection of astrophysical pulses in single-pulse searches, both within and across surveys. We use the output from the first stage of our previously developed two-stage Single-Pulse Event Group IDentification approach and focus on the second stage (i.e., classification of pulse candidates). Specifically, for the first time in time-domain single-pulse searches we (1) use boosting and deep learning algorithms for within-survey classification and (2) investigate cross-survey classification by using two transfer learning methods, trAdaBoost (instance-based) and fine-tuning (parameter-based). Our experimental results are based on two benchmark data sets, Green Bank Telescope Drift-scan (GBTDrift) and Pulsar Arecibo L-band Feed Array (PALFA)-extended, created from the GBTDrift survey and the PALFA survey. The main findings include: (1) Due to the emphasis on high Recall, F4 measure is more appropriate performance indicator than the balanced F1 measure. (2) For the GBTDrift benchmark, AdaBoost outperformed the other models with 98.5% Recall and F4 = 0.942. For the PALFA-extended benchmark, RandomForest performed the best, with 91.6% Recall and F4 = 0.890. (3) Models performance degraded significantly when they were used for pulsar classification across surveys. (4) Transfer learning improved cross-survey classification significantly when training data in the target data set were limited. When the GBTDrift benchmark was used as the target data set, fine-tuned SPEGnet models had the highest F4 measure, while trAdaBoost models had the highest F4 measure when PALFA-extended benchmark was used as the target data set. (5) Pulse classification was affected not only by the between-class imbalance (i.e., pulsar versus non-pulsar), but also by the within-class imbalance, which was more prominent in the case of the PALFA-extended benchmark due to the lack of labeled low signal-to-noise ratio pulsars.

Adaptive weighted over-sampling for imbalanced datasets based on density peaks clustering with heuristic filtering

Learning from imbalanced datasets poses a major challenge in data mining community. When dealing with imbalanced datasets, conventional classification algorithms generally perform poorly as they are originally designed to work under balanced class distribution scenarios. Although there exist different methods to addressing this issue, sampling methods especially over-sampling techniques have shown great potentials as they aim to improve datasets itself rather than the classifiers, which can allow them to be used for any classifier. In this paper, we propose a novel adaptive weighted over-sampling for imbalanced datasets based on density peaks clustering with heuristic filtering. Unlike other clustering-based over-sampling methods, the proposed approach applies modified density peaks clustering rather than traditional k-means clustering techniques to cluster the minority instances due to its capability of accurately identifying sub-clusters with different sizes and densities, which is beneficial for the proposed method to simultaneously accommodate for between-class and within-class imbalance issues caused by various reasons. Subsequently, the size for each identified sub-cluster to be oversampled is adaptively determined according to its own size and density and then the minority instances within each sub-cluster are oversampled based on their probabilities inversely proportional to their distances to the majority class and their densities with the aim of generating more synthetic minority instances for borderline and sparser ones. Finally, in order to avoid the generation of overlapping, a heuristic filtering strategy is also developed to iteratively move the possibly overlapped minority instances away from the majority class. The extensive experimental results on the different imbalanced datasets demonstrate that the proposed approach can achieve better classification performance in most datasets as compared to the other existing over-sampling techniques.

An Imbalanced Malicious Domains Detection Method Based on Passive DNS Traffic Analysis

Although existing malicious domains detection techniques have shown great success in many real-world applications, the problem of learning from imbalanced data is rarely concerned with this day. But the actual DNS traffic is inherently imbalanced; thus how to build malicious domains detection model oriented to imbalanced data is a very important issue worthy of study. This paper proposes a novel imbalanced malicious domains detection method based on passive DNS traffic analysis, which can effectively deal with not only the between-class imbalance problem but also the within-class imbalance problem. The experiments show that this proposed method has favorable performance compared to the existing algorithms.

One-class classification – From theory to practice: A case-study in radioactive threat detection

Over the years, the acceptance of machine learning as a valuable tool in the real-world has caused much interest in the research community; this is particularly the case as the field of Big Data is coming into prominence. However, real-world data comes with a myriad of challenges, amongst the most prominent of which is the fact that it can exhibit a high level of imbalance. This can come in the form of both within- and between-class imbalance. While a significant amount of research has been devoted to the impact of within-class imbalance over binary classifiers, very little attention has been given to their impact on one-class classifiers, which are typically used in situations of extreme between-class imbalance. During our collaboration with Health Canada into the identification of anomalous gamma-ray spectra, the issue of within-class imbalance in a one-class classification setting was highly significant. In this setting, the imbalance comes from the fact that the background data that we wish to model is composed of two concepts (background no-rain and rain); the rain sub-concept is rare and corresponds to spectra affected by the presence of water in the environment. In this article, we present our work into developing systems for detecting anomalous gamma-rays that are able to handle both the inherent between-class and within-class imbalance present in the domain. We test and validate our system over data provided to us by Health Canada from three sites across Canada. Our results indicated that oversampling the sub-concept improves the performance of the baseline classifiers and multiple classifier system when measured by the geometric mean of the per-class accuracy.

Synthetic sampling approach based on model-based clustering for imbalanced data

A dataset exhibits class imbalance problem when one class has very few examples compared to the other class also referred to as between class imbalance. Apart from between-class imbalance, imbalance within classes where classes are composed of different number of sub-clusters with these sub-clusters containing different number of examples may also affect the performance of the classifier. In this paper, we propose a method that can handle both between-class and within-class imbalance simultaneously that also takes into consideration various data intrinsic characteristics. The proposed method uses model-based clustering with respect to classes to identify the sub-clusters present in the dataset and oversamples examples in each sub-cluster in such a manner that it eliminates between class and within class imbalance simultaneously. We validate our approach using neural network on ten publicly available datasets. The experimental results show the proposed method to be statistically significantly superior to other methods.

Synthetic sampling approach based on model-based clustering for imbalanced data

A dataset exhibits class imbalance problem when one class has very few examples compared to the other class also referred to as between class imbalance. Apart from between-class imbalance, imbalance within classes where classes are composed of different number of sub-clusters with these sub-clusters containing different number of examples may also affect the performance of the classifier. In this paper, we propose a method that can handle both between-class and within-class imbalance simultaneously that also takes into consideration various data intrinsic characteristics. The proposed method uses model-based clustering with respect to classes to identify the sub-clusters present in the dataset and oversamples examples in each sub-cluster in such a manner that it eliminates between class and within class imbalance simultaneously. We validate our approach using neural network on ten publicly available datasets. The experimental results show the proposed method to be statistically significantly superior to other methods.

Drug-target interaction prediction via class imbalance-aware ensemble learning.

BackgroundMultiple computational methods for predicting drug-target interactions have been developed to facilitate the drug discovery process. These methods use available data on known drug-target interactions to train classifiers with the purpose of predicting new undiscovered interactions. However, a key challenge regarding this data that has not yet been addressed by these methods, namely class imbalance, is potentially degrading the prediction performance. Class imbalance can be divided into two sub-problems. Firstly, the number of known interacting drug-target pairs is much smaller than that of non-interacting drug-target pairs. This imbalance ratio between interacting and non-interacting drug-target pairs is referred to as the between-class imbalance. Between-class imbalance degrades prediction performance due to the bias in prediction results towards the majority class (i.e. the non-interacting pairs), leading to more prediction errors in the minority class (i.e. the interacting pairs). Secondly, there are multiple types of drug-target interactions in the data with some types having relatively fewer members (or are less represented) than others. This variation in representation of the different interaction types leads to another kind of imbalance referred to as the within-class imbalance. In within-class imbalance, prediction results are biased towards the better represented interaction types, leading to more prediction errors in the less represented interaction types.ResultsWe propose an ensemble learning method that incorporates techniques to address the issues of between-class imbalance and within-class imbalance. Experiments show that the proposed method improves results over 4 state-of-the-art methods. In addition, we simulated cases for new drugs and targets to see how our method would perform in predicting their interactions. New drugs and targets are those for which no prior interactions are known. Our method displayed satisfactory prediction performance and was able to predict many of the interactions successfully.ConclusionsOur proposed method has improved the prediction performance over the existing work, thus proving the importance of addressing problems pertaining to class imbalance in the data.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-1377-y) contains supplementary material, which is available to authorized users.

KRNN: k Rare-class Nearest Neighbour classification

Imbalanced classification is a challenging problem. Re-sampling and cost-sensitive learning are global strategies for generality-oriented algorithms such as the decision tree, targeting inter-class imbalance. We research local strategies for the specificity-oriented learning algorithms like the k Nearest Neighbour (KNN) to address the within-class imbalance issue of positive data sparsity. We propose an algorithm k Rare-class Nearest Neighbour, or KRNN, by directly adjusting the induction bias of KNN. We propose to form dynamic query neighbourhoods, and to further adjust the positive posterior probability estimation to bias classification towards the rare class. We conducted extensive experiments on thirty real-world and artificial datasets to evaluate the performance of KRNN. Our experiments showed that KRNN significantly improved KNN for classification of the rare class, and often outperformed re-sampling and cost-sensitive learning strategies with generality-oriented base learners.

Hybrid probabilistic sampling with random subspace for imbalanced data learning

Class imbalance is one of the challenging problems for machine learning in many real-world applications. Other issues, such as within-class imbalance and high dimensionality, can exacerbate the problem. We propose a method HPS-DRS that combines two ideas: Hybrid Probabilistic Sampling technique ensemble with Diverse Random Subspace to address these issues. HPS improves the performance of traditional re-sampling algorithms with the aid of probability function, since it is not sufficient to simply manipulate the class sizes for imbalanced data with complex distribution. Moreover, DRS ensemble employs the minimum overlapping mechanism to provide diversity and weighted voting, so as to improve the generalization performance. The experimental results demonstrate that our method is efficient for learning from imbalanced data and can achieve better results than state-of-the-art methods for imbalanced data.

Ensemble-based hybrid probabilistic sampling for imbalanced data learning in lung nodule CAD

Classification plays a critical role in false positive reduction (FPR) in lung nodule computer aided detection (CAD). The difficulty of FPR lies in the variation of the appearances of the nodules, and the imbalance distribution between the nodule and non-nodule class. Moreover, the presence of inherent complex structures in data distribution, such as within-class imbalance and high-dimensionality are other critical factors of decreasing classification performance. To solve these challenges, we proposed a hybrid probabilistic sampling combined with diverse random subspace ensemble. Experimental results demonstrate the effectiveness of the proposed method in terms of geometric mean (G-mean) and area under the ROC curve (AUC) compared with commonly used methods.