Algorithm-level Methods Research Articles

CTGAN-ENN: a tabular GAN-based hybrid sampling method for imbalanced and overlapped data in customer churn prediction

Class imbalance is one of many problems of customer churn datasets. One of the common problems is class overlap, where the data have a similar instance between classes. The prediction task of customer churn becomes more challenging when there is class overlap in the data training. In this research, we suggested a hybrid method based on tabular GANs, called CTGAN-ENN, to address class overlap and imbalanced data in datasets of customers that churn. We used five different customer churn datasets from an open platform. CTGAN is a tabular GAN-based oversampling to address class imbalance but has a class overlap problem. We combined CTGAN with the ENN under-sampling technique to overcome the class overlap. CTGAN-ENN reduced the number of class overlaps by each feature in all datasets. We investigated how effective CTGAN-ENN is in each machine learning technique. Based on our experiments, CTGAN-ENN achieved satisfactory results in KNN, GBM, XGB and LGB machine learning performance for customer churn predictions. We compared CTGAN-ENN with common over-sampling and hybrid sampling methods, and CTGAN-ENN achieved outperform results compared with other sampling methods and algorithm-level methods with cost-sensitive learning in several machine learning algorithms. We provide a time consumption algorithm between CTGAN and CTGAN-ENN. CTGAN-ENN achieved less time consumption than CTGAN. Our research work provides a new framework to handle customer churn prediction problems with several types of imbalanced datasets and can be useful in real-world data from customer churn prediction.

An extended belief rule-based system with hybrid sampling strategy for imbalanced rule base

Imbalanced dataset is an important focus for classification. As the mainstream of addressing imbalanced dataset, data-level methods are trapped in facing difficultly-determined subjective parameters, and the inconsistency between new minority samples and original minority samples simultaneously. To address it, this paper develops an extended belief-rule-based (EBRB) system with hybrid sampling strategy, which is a white-box classifier. The hybrid sampling strategy is composed of an under-sampling process and an oversampling process, in which subjective parameters are not involved. The under-sampling is to identify and remove overlapping majority rules by iteratively determining an appropriate objective threshold for calculating the inconsistency degree of rule base, and to determine and remove redundant non-overlapping majority rules by using the density of non-overlapping rules in clustered groups. The oversampling is to design a differential evolution based iterative process to generate new minority rules in groups by minimizing the inconsistency of rule base. The distribution of original dataset is maintained extremely by balancing rules in clustered majority and minority groups, respectively. This EBRB system is used for the auxiliary diagnosis of thyroid nodules, and its superior performance is highlighted by the comparison with existing EBRB systems, representative data-level methods, and algorithm-level methods.

A Survey on Classifying Big Data with Label Noise

Class label noise is a critical component of data quality that directly inhibits the predictive performance of machine learning algorithms. While many data-level and algorithm-level methods exist for treating label noise, the challenges associated with big data call for new and improved methods. This survey addresses these concerns by providing an extensive literature review on treating label noise within big data. We begin with an introduction to the class label noise problem and traditional methods for treating label noise. Next, we present 30 methods for treating class label noise in a range of big data contexts, i.e., high-volume, high-variety, and high-velocity problems. The surveyed works include distributed solutions capable of operating on datasets of arbitrary sizes, deep learning techniques for large-scale datasets with limited clean labels, and streaming techniques for detecting class noise in the presence of concept drift. Common trends and best practices are identified in each of these areas, implementation details are reviewed, empirical results are compared across studies when applicable, and references to 17 open-source projects and programming packages are provided. An emphasis on label noise challenges, solutions, and empirical results as they relate to big data distinguishes this work as a unique contribution that will inspire future research and guide machine learning practitioners.

An imbalanced binary classification method based on contrastive learning using multi-label confidence comparisons within sample-neighbors pair

For imbalanced classification, data-level methods can achieve inter-class balance, but the samples generated do not contain new information and cannot avoid the problem of introducing noise. Algorithm-level methods may lead to overfitting of the model, and its classification effect is more dependent on the specific dataset and classification task, which means they lack universality. In addition, how to deeply mine the differences in the distribution of data overlap areas, and how to effectively mine the differences between categories when the absolute number of minority samples is small, are also important challenges in imbalanced classification. This paper proposes an imbalanced binary classification method using multi-label confidence comparisons based on contrastive learning. Different from the previous idea of directly learning its distribution characteristics from minority samples, combined with the idea of contrastive learning, the classification task is redefined as the multi-label matching task by mining the deep features that can represent the commonality and difference between the neighboring samples. Multiple differentiated contrastive sample groups are obtained through random sampling in its neighbor sample pool for each sample. This sample is combined with its contrastive sample groups to form multiple sample-neighbor pairs as training samples in the multi-label matching task. The original dataset is multiplied without introducing noise, laying a foundation for the effective mining of class differences when the absolute number of minority class samples is small. Based on the corresponding reconstruction error generated by Variational AutoEncoder (VAE), for sample-neighbor pairs, a multi-label matching loss between target samples and contrastive sample groups that integrates the idea of contrastive learning is designed. a robust classifier is obtained through simultaneous iterative learning of reconstruction error and multi-label matching loss, which can better mine the distribution differences of overlapping regions. In the testing phase, multiple different contrastive sample groups and the corresponding prediction results of the samples to be classified are obtained, which categories can be judged by integrating the predictions of each group for reverse reasoning. Experimental results on 38 public datasets show that the method outperforms typical imbalanced classification methods in both F1-measure and G-mean.

A Hybrid Algorithm-Level Ensemble Model for Imbalanced Credit Default Prediction in the Energy Industry

Credit default prediction for the energy industry is essential to promoting the healthy development of the energy industry in China. While previous studies have constructed various credit default prediction models with brilliant performance, the class-imbalance problem in the credit default dataset cannot be ignored, where the numbers of credit default cases are usually much smaller than the number of non-default ones. To address the class-imbalance problem, we proposed a novel CT-XGBoost model, which adds to XGBoost with two algorithm-level methods for class imbalance, including the cost-sensitive strategy and threshold method. Based on the credit default dataset consisting of energy corporates in western China, which suffers from the class-imbalance problem, the CT-XGBoost model achieves better performance than the conventional models. The results indicate that the proposed model can efficiently alleviate the inherent class-imbalance problem in the credit default dataset. Moreover, we analyze how the prediction performance is influenced by different parameter settings in the cost-sensitive strategy and threshold method. This study can help market investors and regulators precisely assess the credit risk in the energy industry and provides theoretical guidance to solving the class-imbalance problem in credit default prediction.

Cost-Sensitive Variational Autoencoding Classifier for Imbalanced Data Classification

Classification is among the core tasks in machine learning. Existing classification algorithms are typically based on the assumption of at least roughly balanced data classes. When performing tasks involving imbalanced data, such classifiers ignore the minority data in consideration of the overall accuracy. The performance of traditional classification algorithms based on the assumption of balanced data distribution is insufficient because the minority-class samples are often more important than others, such as positive samples, in disease diagnosis. In this study, we propose a cost-sensitive variational autoencoding classifier that combines data-level and algorithm-level methods to solve the problem of imbalanced data classification. Cost-sensitive factors are introduced to assign a high cost to the misclassification of minority data, which biases the classifier toward minority data. We also designed misclassification costs closely related to tasks by embedding domain knowledge. Experimental results show that the proposed method performed the classification of bulk amorphous materials well.

Handling Class Imbalance in Online Transaction Fraud Detection

With the rise of internet facilities, a greater number of people have started doing online transactions at an exponential rate in recent years as the online transaction system has eliminated the need of going to the bank physically for every transaction. However, the fraud cases have also increased causing the loss of money to the consumers. Hence, an effective fraud detection system is the need of the hour which can detect fraudulent transactions automatically in real-time. Generally, the genuine transactions are large in number than the fraudulent transactions which leads to the class imbalance problem. In this research work, an online transaction fraud detection system using deep learning has been proposed which can handle class imbalance problem by applying algorithm-level methods which modify the learning of the model to focus more on the minority class i.e., fraud transactions. A novel loss function named Weighted Hard- Reduced Focal Loss (WH-RFL) has been proposed which has achieved maximum fraud detection rate i.e., True Positive Rate (TPR) at the cost of misclassification of few genuine transactions as high TPR is preferred over a high True Negative Rate (TNR) in fraud detection system and same has been demonstrated using three publicly available imbalanced transactional datasets. Also, Thresholding has been applied to optimize the decision threshold using cross-validation to detect maximum number of frauds and it has been demonstrated by the experimental results that the selection of the right thresholding method with deep learning yields better results.

Classification of imbalanced oral cancer image data from high-risk population.

.Significance: Early detection of oral cancer is vital for high-risk patients, and machine learning-based automatic classification is ideal for disease screening. However, current datasets collected from high-risk populations are unbalanced and often have detrimental effects on the performance of classification.Aim: To reduce the class bias caused by data imbalance.Approach: We collected 3851 polarized white light cheek mucosa images using our customized oral cancer screening device. We use weight balancing, data augmentation, undersampling, focal loss, and ensemble methods to improve the neural network performance of oral cancer image classification with the imbalanced multi-class datasets captured from high-risk populations during oral cancer screening in low-resource settings.Results: By applying both data-level and algorithm-level approaches to the deep learning training process, the performance of the minority classes, which were difficult to distinguish at the beginning, has been improved. The accuracy of “premalignancy” class is also increased, which is ideal for screening applications.Conclusions: Experimental results show that the class bias induced by imbalanced oral cancer image datasets could be reduced using both data- and algorithm-level methods. Our study may provide an important basis for helping understand the influence of unbalanced datasets on oral cancer deep learning classifiers and how to mitigate.

Deep balanced cascade forest: An novel fault diagnosis method for data imbalance

Data imbalance is a common problem in rotating machinery fault diagnosis. Traditional data-driven diagnosis methods, which learn fault features based on balance dataset, would be significantly affected by imbalanced data. In this paper, a novel imbalanced data related fault diagnosis method named deep balanced cascade forest is proposed to solve this problem. Deep balanced cascade forest is a multi-channel cascade forest, in which, each of its channels adaptively generates deep cascade structure and is trained on independent data. To enhance the performance of imbalance classification, the deep balanced cascade forest is promoted from both aspects of resampling and algorithm design. A hybrid sampling method, namely Up–down Sampling, is proposed to provide rebalanced data for each cascade forest channel. Meanwhile, a new type of balanced forest with an improved balanced information entropy for attribute selection is designed as the basic classifier of cascade forest. The good synergy of these two methods is the key to the deep balanced cascade forest model. This good synergy makes deep balanced cascade forest achieve the fusion of data-level methods and algorithm-level methods. Comparative experiments on sufficient imbalanced datasets have been designed to verify the performance of the proposed model, and results confirm that deep balanced cascade forest is much more stable and effective in handling imbalance fault diagnosis problem compared to the popular deep learning methods.

A minority oversampling approach for fault detection with heterogeneous imbalanced data

Between-class imbalance and feature heterogeneity commonly coexist in monitoring data collected from engineering systems. The decision hyperplanes of data-driven methods when adopted for fault detection with imbalanced data may be biased to the majority class, resulting in a low fault-detection rate. Various data- and algorithm-level methods have been proposed, with minority oversampling methods among the most popular and successful. However, state-of-the-art minority oversampling methods are unsuitable for imbalanced data with heterogeneous features, including both numeric and nominal variables. There are two main drawbacks: 1) taking a nominal variable as a numeric variable is not trivial, and synthetic minority samples may exceed the value range of nominal variables; 2) conventional distance measures, e.g., Euclidean distance, cannot properly measure the similarity of samples with heterogeneous features. For these considerations, this work proposes new fault-detection methods. The methodological contributions include: 1) two different distance measures adopted for heterogeneous features in the minority class; 2) a new method for coordinate calculation of synthetic samples considering feature heterogeneity; and 3) a new strategy to encode nominal variables into numeric data for data-driven models. Several public heterogeneous imbalanced datasets and a real case study considering fault detection in high-speed trains are considered to verify the effectiveness of the proposed methods. To the knowledge of the author, this is also the first time that the effectiveness of diverse oversampling methods on heterogeneous imbalance data are specifically discussed.

Fusion Algorithm for Imbalanced EEG Data Processing in Seizure Detection

Purpose: Seizure detection algorithms (SDAs) based on electroencephalography (EEG) have been described in previous studies, but the imbalanced data distribution of ictal and interictal states continue to pose a technical challenge. This study proposes a novel algorithm to address the imbalanced classification problem and improve seizure detection performance. Method: The proposed algorithm is designed based on hybrid sampling and a cost-sensitive (CS) classifier. Hybrid sampling resamples the imbalanced EEG data at data-level, and the CS classifier, which is used as an algorithm-level tool, reduces the overall misclassification cost of seizure detection. The synthetic minority oversampling technique and undersampling TomekLink technique are combined to reduce the imbalanced ratio between ictal and interictal states while retaining the generalization ability. Finally, CS support vector machine classifies the resampled EEG feature vectors, assigning different cost sensitive parameters to moderate the poor performance resulting from the imbalanced distribution problem. Result: The proposed algorithm improved the average sensitivity and AUC by 46.67% and 0.0482, respectively, compared with the original results without using the imbalanced EEG data processing (IEDP) technique. Experimental results showed that an average sensitivity and AUC of 86.34% and 0.9837, respectively, could be obtained across all cases. Finally, a performance evaluation showed that the proposed algorithm outperformed published methods in terms seizure detection. Conclusion: A fusion algorithm combining data- and algorithm-level methods can achieve high sensitivity and AUC compared with existing IEDP methods. Thus, SDA performance can be improved, enabling their clinical use with EEG-based SMSs.

Robust cost-sensitive kernel method with Blinex loss and its applications in credit risk evaluation

Credit risk evaluation is a crucial yet challenging problem in financial analysis. It can not only help institutions reduce risk and ensure profitability, but also improve consumers’ fair practices. The data-driven algorithms such as artificial intelligence techniques regard the evaluation as a classification problem and aim to classify transactions as default or non-default. Since non-default samples greatly outnumber default samples, it is a typical imbalanced learning problem and each class or each sample needs special treatment. Numerous data-level, algorithm-level and hybrid methods are presented, and cost-sensitive support vector machines (CSSVMs) are representative algorithm-level methods. Based on the minimization of symmetric and unbounded loss functions, CSSVMs impose higher penalties on the misclassification costs of minority instances using domain specific parameters. However, such loss functions as error measurement cannot have an obvious cost-sensitive generalization. In this paper, we propose a robust cost-sensitive kernel method with Blinex loss (CSKB), which can be applied in credit risk evaluation. By inheriting the elegant merits of Blinex loss function, i.e., asymmetry and boundedness, CSKB not only flexibly controls distinct costs for both classes, but also enjoys noise robustness. As a data-driven decision-making paradigm of credit risk evaluation, CSKB can achieve the “win-win” situation for both the financial institutions and consumers. We solve linear and nonlinear CSKB by Nesterov accelerated gradient algorithm and Pegasos algorithm respectively. Moreover, the generalization capability of CSKB is theoretically analyzed. Comprehensive experiments on synthetic, UCI and credit risk evaluation datasets demonstrate that CSKB compares more favorably than other benchmark methods in terms of various measures.

Deep Attention-Based Imbalanced Image Classification.

Class imbalance is a common problem in real-world image classification problems, some classes are with abundant data, and the other classes are not. In this case, the representations of classifiers are likely to be biased toward the majority classes and it is challenging to learn proper features, leading to unpromising performance. To eliminate this biased feature representation, many algorithm-level methods learn to pay more attention to the minority classes explicitly according to the prior knowledge of the data distribution. In this article, an attention-based approach called deep attention-based imbalanced image classification (DAIIC) is proposed to automatically pay more attention to the minority classes in a data-driven manner. In the proposed method, an attention network and a novel attention augmented logistic regression function are employed to encapsulate as many features, which belongs to the minority classes, as possible into the discriminative feature learning process by assigning the attention for different classes jointly in both the prediction and feature spaces. With the proposed object function, DAIIC can automatically learn the misclassification costs for different classes. Then, the learned misclassification costs can be used to guide the training process to learn more discriminative features using the designed attention networks. Furthermore, the proposed method is applicable to various types of networks and data sets. Experimental results on both single-label and multilabel imbalanced image classification data sets show that the proposed method has good generalizability and outperforms several state-of-the-art methods for imbalanced image classification.

A weighted hybrid ensemble method for classifying imbalanced data

In real datasets, most are unbalanced. Data imbalance can be defined as the number of instances in some classes greatly exceeds the number of instances in other classes. Whether in the field of data mining or machine learning, data imbalance can have adverse effects. At present, the methods to solve the problem of data imbalance can be divided into data-level methods, algorithm-level methods and hybrid methods. In this paper, we propose a weighted hybrid ensemble method for classifying imbalanced data in binary classification tasks, called WHMBoost. In the framework of the boosting algorithm, the presented method combines two data sampling methods and two base classifiers, and each sampling method and each base classifier is assigned corresponding weights, which makes them have better complementary advantages. The performance of WHMBoost has been evaluated on 40 benchmark imbalanced datasets with state of the art ensemble methods like AdaBoost, RUSBoost, SMOTEBoost using AUC, F-Measure and Geometric Mean as the performance evaluation criteria. Experimental results show significant improvement over the other methods and it can be concluded that WHMBoost is a promising and effective algorithm to deal with imbalance datasets.

Hybrid Machine Intelligence for Imbalanced Data

Data Imbalance is the major problem faced while implementing machine learning concepts to real life problems. Imbalanced dataset poses a severe issue for a huge many supervised learning problems which leads to increasing interest from both industrial and academic sectors. Since the class having less instances of data points is outnumbered by the class having maximum instances of data points. The training dataset includes all data points to fit into a classical classifier. The classifier classifies the whole set of data to majority class by and ignores the minority class by considering it as noise. Therefore it is significant to choose a training dataset which perfectly fits in the processing phase of the order of an imbalanced dataset. Imbalance data is present everywhere in the real world. Existing studies on imbalanced data inquiries them for binary classification. It is presumed that conventional ML techniques try to reduce experimental risk factors. Therefore, the arrangement precision of the minority class is regularly abandoned. Different data level methods like undersampling, oversampling and algorithm level methods like one class learning and ensemble cost sensitive have been presented for better performance of classifiers while handling imbalanced datasets and distributions. Grounded on these factors, this study aimed a hybrid viewpoint to handle the problems caused by imbalance data in a better manner.

An Imbalanced R-STDP Learning Rule in Spiking Neural Networks for Medical Image Classification

Spiking neural networks (SNNs) have the advantages of inherent power-efficiency, biological plausibility and good image recognition performance. They are good candidates for medical image classification especially when the labeled training data are limited. In medical image classification, one of the major challenges is the highly class imbalanced problem which causes deep learning networks to bias towards the majority class and poorly recognizes the minority class. Despite that there are some methods for addressing this problem, very few algorithm-level methods exist for SNNs. In this work, we propose an imbalanced reward-modulation spike-timing-dependent plasticity (R-STDP) learning rule for SNNs to solve the medical image class imbalanced problem. We introduce an imbalanced reward coefficient for the R-STDP learning rule to set the reward from the minority class to be higher than that of the majority class, and this reward coefficient can help to set the class-dependent rewards according to the data statistic of the training dataset. Experiment results on three benchmark datasets with imbalanced splits show that our method significantly improves the performance than that of the baseline SNNs and outperforms the compared state-of-the-art methods addressing medical image class imbalanced problem including data-level and algorithm-level methods. Moreover, our method achieves excellent classification performance on the imbalanced medical dataset ISIC-2018. The results show that the proposed method can well help SNNs in classifying imbalanced medical image datasets. Besides, our proposed method can obtain high sensitivity to disease class by adjusting the reward coefficient, which is very useful for identifying disease samples in medical diagnostic tasks.

Medicare fraud detection using neural networks

Access to affordable healthcare is a nationwide concern that impacts a large majority of the United States population. Medicare is a Federal Government healthcare program that provides affordable health insurance to the elderly population and individuals with select disabilities. Unfortunately, there is a significant amount of fraud, waste, and abuse within the Medicare system that costs taxpayers billions of dollars and puts beneficiaries’ health and welfare at risk. Previous work has shown that publicly available Medicare claims data can be leveraged to construct machine learning models capable of automating fraud detection, but challenges associated with class-imbalanced big data hinder performance. With a minority class size of 0.03% and an opportunity to improve existing results, we use the Medicare fraud detection task to compare six deep learning methods designed to address the class imbalance problem. Data-level techniques used in this study include random over-sampling (ROS), random under-sampling (RUS), and a hybrid ROS–RUS. The algorithm-level techniques evaluated include a cost-sensitive loss function, the Focal Loss, and the Mean False Error Loss. A range of class ratios are tested by varying sample rates and desirable class-wise performance is achieved by identifying optimal decision thresholds for each model. Neural networks are evaluated on a 20% holdout test set, and results are reported using the area under the receiver operating characteristic curve (AUC). Results show that ROS and ROS–RUS perform significantly better than baseline and algorithm-level methods with average AUC scores of 0.8505 and 0.8509, while ROS–RUS maximizes efficiency with a 4× speedup in training time. Plain RUS outperforms baseline methods with up to 30× improvements in training time, and all algorithm-level methods are found to produce more stable decision boundaries than baseline methods. Thresholding results suggest that the decision threshold always be optimized using a validation set, as we observe a strong linear relationship between the minority class size and the optimal threshold. To the best of our knowledge, this is the first study to compare multiple data-level and algorithm-level deep learning methods across a range of class distributions. Additional contributions include a unique analysis of the relationship between minority class size and optimal decision threshold and state-of-the-art performance on the given Medicare fraud detection task.

A survey on addressing high-class imbalance in big data

In a majority–minority classification problem, class imbalance in the dataset(s) can dramatically skew the performance of classifiers, introducing a prediction bias for the majority class. Assuming the positive (minority) class is the group of interest and the given application domain dictates that a false negative is much costlier than a false positive, a negative (majority) class prediction bias could have adverse consequences. With big data, the mitigation of class imbalance poses an even greater challenge because of the varied and complex structure of the relatively much larger datasets. This paper provides a large survey of published studies within the last 8 years, focusing on high-class imbalance (i.e., a majority-to-minority class ratio between 100:1 and 10,000:1) in big data in order to assess the state-of-the-art in addressing adverse effects due to class imbalance. In this paper, two techniques are covered which include Data-Level (e.g., data sampling) and Algorithm-Level (e.g., cost-sensitive and hybrid/ensemble) Methods. Data sampling methods are popular in addressing class imbalance, with Random Over-Sampling methods generally showing better overall results. At the Algorithm-Level, there are some outstanding performers. Yet, in the published studies, there are inconsistent and conflicting results, coupled with a limited scope in evaluated techniques, indicating the need for more comprehensive, comparative studies.

Learning from imbalanced data: open challenges and future directions

Despite more than two decades of continuous development learning from imbalanced data is still a focus of intense research. Starting as a problem of skewed distributions of binary tasks, this topic evolved way beyond this conception. With the expansion of machine learning and data mining, combined with the arrival of big data era, we have gained a deeper insight into the nature of imbalanced learning, while at the same time facing new emerging challenges. Data-level and algorithm-level methods are constantly being improved and hybrid approaches gain increasing popularity. Recent trends focus on analyzing not only the disproportion between classes, but also other difficulties embedded in the nature of data. New real-life problems motivate researchers to focus on computationally efficient, adaptive and real-time methods. This paper aims at discussing open issues and challenges that need to be addressed to further develop the field of imbalanced learning. Seven vital areas of research in this topic are identified, covering the full spectrum of learning from imbalanced data: classification, regression, clustering, data streams, big data analytics and applications, e.g., in social media and computer vision. This paper provides a discussion and suggestions concerning lines of future research for each of them.

Adaptive Sampling with Optimal Cost for Class-Imbalance Learning

Learning from imbalanced data sets is one of the challenging problems in machine learning, which means the number of negative examples is far more than that of positive examples. The main problems of existing methods are: (1) The degree of re-sampling, a key factor greatly affecting performance, needs to be pre-fixed, which is difficult to make the optimal choice; (2) Many useful negative samples are discarded in under-sampling; (3) The effectiveness of algorithm-level methods are limited because they just use the original training data for single classifier. To address the above issues, a novel approach of adaptive sampling with optimal cost is proposed for class-imbalance learning in this paper. The novelty of the proposed approach mainly lies in: adaptively over-sampling the minority positive examples and under-sampling the majority negative examples, forming different sub-classifiers by different subsets of training data with the best cost ratio adaptively chosen, and combining these sub-classifiers according to their accuracy to create a strong classifier. It aims to make full use of the whole training data and improve the performance of class-imbalance learning classifier. The solid experiments are conducted to compare the performance between the proposed approach and 12 state-of-the-art methods on challenging 16 UCI data sets on 3 evaluation metrics, and the results show the proposed approach can achieve superior performance in class-imbalance learning.