Imbalanced Data Classification Problem Research Articles

Deep convolutional neural networks with genetic algorithm-based synthetic minority over-sampling technique for improved imbalanced data classification

Imbalanced data classification presents a challenge in machine learning, inducing biased model learning. Moreover, data dimensionality poses another challenge as it highly impacts classifier performance. This paper proposes a new deep-learning method that combines feature selection with oversampling to address these challenges. The proposed approach, GA-SMOTE-DCNN, integrates a genetic algorithm (GA) for feature selection, SMOTE for oversampling, and a deep 1D-convolutional neural network (DCNN) for classification. This study reveals that pre-splitting the data into training and testing sets before applying SMOTE results in higher accuracy, showing an improvement in accuracy ranging between 1.94% and 3.98% compared to post-SMOTE splitting for each dataset. This method achieved accuracy rates of 86.81% for the Balance Scale dataset, 86.15% for the Oil Spill dataset, 89.21% for the Yeast dataset, 91.32% for the Mammography dataset, 88.23% for the Australian credit dataset, and 89.53% for the German Credit dataset when compared with benchmark methods, underscoring its significance in tackling high-dimensional and imbalanced data classification problems. This method demonstrates scalability in effectively addressing challenges associated with high-dimensional and imbalanced data classification across various domains.

Edge computing-based proactive control method for industrial product manufacturing quality prediction

With the emergence of intelligent manufacturing, new-generation information technologies such as big data and artificial intelligence are rapidly integrating with the manufacturing industry. One of the primary applications is to assist manufacturing plants in predicting product quality. Traditional predictive models primarily focus on establishing high-precision classification or regression models, with less emphasis on imbalanced data. This is a specific but common scenario in practical industrial environments concerning quality prediction. A SMOTE-XGboost quality prediction active control method based on joint optimization hyperparameters is proposed to address the problem of imbalanced data classification in product quality prediction. In addition, edge computing technology is introduced to address issues in industrial manufacturing, such as the large bandwidth load and resource limitations associated with traditional cloud computing models. Finally, the practicality and effectiveness of the proposed method are validated through a case study of the brake disc production line. Experimental results indicate that the proposed method outperforms other classification methods in brake disc quality prediction.

A hybrid multi-criteria meta-learner based classifier for imbalanced data

Numerous imbalanced datasets exist in modern machine learning dilemmas. Challenges of generalization and fairness stem from the existence of underrepresented classes with sensitive characteristics. Additionally, defining an optimal criterion is crucial when dealing with imbalanced data. To address this, it is common in practice to employ composite metrics and loss functions that evenly distribute the cost across each class.However, relying solely on a single metric or loss function, frequently yields sub-par outcomes due to the tendency of the optimization criteria to be overly responsive to imbalanced data, resulting in excessive adaptation and flawed generalization. In order to overcome these limitations, various heuristics can be applied to optimize the criterion.To this end, we aim at addressing the problem of imbalanced data classification by leveraging a multicriteria ensemble procedure, the Hybrid Multi-criteria Meta-learner (HML). The suggested approach focuses on optimizing precision, recall together with balanced accuracy in the Multi-Objective optimization phase and brings forth optimal ensembles, from which the concluding Meta-learner can be chosen after a clustering step, based on a multicriteria decision-making aspect contingent on the selected metrics. Comprehensive experiments portrayed the effectiveness of HML, offering both quality assurance and addresses a vast array of imbalanced data problems.

A multi-manifold learning based instance weighting and under-sampling for imbalanced data classification problems

Under-sampling is a technique to overcome imbalanced class problem, however, selecting the instances to be dropped and measuring their informativeness is an important concern. This paper tries to bring up a new point of view in this regard and exploit the structure of data to decide on the importance of the data points. For this purpose, a multi-manifold learning approach is proposed. Manifolds represent the underlying structures of data and can help extract the latent space for data distribution. However, there is no evidence that we can rely on a single manifold to extract the local neighborhood of the dataset. Therefore, this paper proposes an ensemble of manifold learning approaches and evaluates each manifold based on an information loss-based heuristic. Having computed the optimality score of each manifold, the centrality and marginality degrees of samples are computed on the manifolds and weighted by the corresponding score. A gradual elimination approach is proposed, which tries to balance the classes while avoiding a drop in the F measure on the validation dataset. The proposed method is evaluated on 22 imbalanced datasets from the KEEL and UCI repositories with different classification measures. The results of the experiments demonstrate that the proposed approach is more effective than other similar approaches and is far better than the previous approaches, especially when the imbalance ratio is very high.

Evolutionary Weighted Broad Learning and Its Application to Fault Diagnosis in Self-Organizing Cellular Networks.

As a novel neural network-based learning framework, a broad learning system (BLS) has attracted much attention due to its excellent performance on regression and balanced classification problems. However, it is found to be unsuitable for imbalanced data classification problems because it treats each class in an imbalanced dataset equally. To address this issue, this work proposes a weighted BLS (WBLS) in which the weight assigned to each class depends on the number of samples in it. In order to further boost its classification performance, an improved differential evolution algorithm is proposed to automatically optimize its parameters, including the ones in BLS and newly generated weights. We first optimize the parameters with a training dataset, and then apply them to WBLS on a test dataset. The experiments on 20 imbalanced classification problems have shown that our proposed method can achieve higher classification accuracy than the other methods in terms of several widely used performance metrics. Finally, it is applied to fault diagnosis in self-organizing cellular networks to further show its applicability to industrial application problems.

Imbalanced Text Sentiment Classification Based on Multi-Channel BLTCN-BLSTM Self-Attention.

With the continuous expansion of the field of natural language processing, researchers have found that there is a phenomenon of imbalanced data distribution in some practical problems, and the excellent performance of most methods is based on the assumption that the samples in the dataset are data balanced. Therefore, the imbalanced data classification problem has gradually become a problem that needs to be studied. Aiming at the sentiment information mining of an imbalanced short text review dataset, this paper proposed a fusion multi-channel BLTCN-BLSTM self-attention sentiment classification method. By building a multi-channel BLTCN-BLSTM self-attention network model, the sample after word embedding processing is used as the input of the multi-channel, and after fully extracting features, the self-attention mechanism is fused to strengthen the sentiment to further fully extract text features. At the same time, focus loss rebalancing and classifier enhancement are combined to realize text sentiment predictions. The experimental results show that the optimal F1 value is up to 0.893 on the Chnsenticorp-HPL-10,000 corpus. The comparison and ablation of experimental results, including accuracy, recall, and F1-measure, show that the proposed model can fully integrate the weight of emotional feature words. It effectively improves the sentiment classification performance of imbalanced short-text review data.

Sampling strategies for handling data imbalance problem: An Extensive Review

The imbalanced data classification is a major issue in data mining. Many researchers have proposed various solutions which addressed imbalanced data problem which is broadly categorized into data level and algorithm level. Class distributions are adjusted in data level method. Creating an algorithm or modifying the existing algorithm is an appropriate approach used in algorithm level method. Imbalanced data classification problem can be resolved by means of Sampling, Random over sampling, Random under sampling, Resampling and by SMOTE (Synthetic Minority Oversampling Techniques). Resampling includes k-means clustering, density-based clustering, neural networks and ensemble. However, no algorithm or a method has an ability to remove bias in data classification, thereby integration of kernel methods with sampling methods or integration of sampling and boosting methods or integration Kernel based with Support Vector Machines (SVM) need to be performed a great extent to get the desired accuracy and performance. The main objective of this paper is to focus on various sampling strategies that are based on sampling and resampling methods and improving the concept of learning within class imbalanced data. It also explains the objectives of the models used by several researchers and emphasized the performance along with the outcomes.

Parallel Metaheuristic Algorithms for Solving Imbalanced Data Classification Problems

Parallel Metaheuristic Algorithms for Solving Imbalanced Data Classification Problems

Locally alignment based manifold learning for simultaneous feature selection and extraction in classification problems

Dimensionality reduction is an important step in increasing the performance of machine learning algorithms while decreasing the processing time. From feature reduction approaches, feature extraction is aimed to achieve better data representation while feature selection is meant to discard redundant features. The main contribution of this manuscript is to propose a hybrid feature selection and extraction approach which performs these tasks in a unified framework, simultaneously. Also, the proposed approach aims to maintain the manifold of data using locally alignment constrains. The main goal of these suggestions is to discriminate different classes while reducing the redundancy of the samples specially for the imbalanced data classification problems. The proposed approach optimizes a hybrid objective function that tries to both increase the between class discrimination and decrease the within class distribution while minimizing the information loss. Also, we have embedded a weighting factor into the objective function to achieve a basis for feature importance measurement and feature selection. To evaluate the proposed approach, a multiclass linear SVM classifier is applied on the reduced data in a k-fold cross-validation scheme, and the accuracy, as well as F-score, is used as the performance measure. Comparisons of the proposed method with some recent approaches on 12 datasets of UCI, show the superiority of the proposed method over previously proposed approaches. Also, Friedman and Nemenyi tests are applied which show significant improvement in the proposed approach.

Chaos follow the leader algorithm: Application to data classification

Classification is the process of systematically arranging the data into multiple groups, which gives a better understanding of large datasets. Considerable research has been carried out on data classification to develop a predictive model with high accuracy. This paper introduces chaos in the Follow the leader (FTL) optimization algorithm to achieve global optima for imbalanced data classification problems. In the FTL algorithm, the random parameter ω may not give enough exploration to the particles, which may lead to local stagnation problems. One of the solutions to this problem can be the replacement of the random parameter ω by a chaotic variable. For a detailed study on the chaotic behavior of ω in FTL, the algorithm is implemented with ten different chaotic maps and tested on twenty-four standard benchmark functions. Thereafter, the best chaos FTL (cFTL) algorithm is tested on six different complex real engineering problems and compared with ten meta-heuristic algorithms. Moreover, to further validate the performance of cFT, it has been applied to ten standard classification datasets collected from UCI and Kaggle data repositories. The obtained results show that cFTL outperforms FTL and other meta-heuristic algorithms.

SORAG: Synthetic Data Over-Sampling Strategy on Multi-Label Graphs

In many real-world networks of interest in the field of remote sensing (e.g., public transport networks), nodes are associated with multiple labels, and node classes are imbalanced; that is, some classes have significantly fewer samples than others. However, the research problem of imbalanced multi-label graph node classification remains unexplored. This non-trivial task challenges the existing graph neural networks (GNNs) because the majority class can dominate the loss functions of GNNs and result in the overfitting of the majority class features and label correlations. On non-graph data, minority over-sampling methods (such as the synthetic minority over-sampling technique and its variants) have been demonstrated to be effective for the imbalanced data classification problem. This study proposes and validates a new hypothesis with unlabeled data over-sampling, which is meaningless for imbalanced non-graph data; however, feature propagation and topological interplay mechanisms between graph nodes can facilitate the representation learning of imbalanced graphs. Furthermore, we determine empirically that ensemble data synthesis through the creation of virtual minority samples in the central region of a minority and generation of virtual unlabeled samples in the boundary region between a minority and majority is the best practice for the imbalanced multi-label graph node classification task. Our proposed novel data over-sampling framework is evaluated using multiple real-world network datasets, and it outperforms diverse, strong benchmark models by a large margin.

Research on imbalance machine learning methods for MRT_1WI soft tissue sarcoma data

BackgroundSoft tissue sarcoma is a rare and highly heterogeneous tumor in clinical practice. Pathological grading of the soft tissue sarcoma is a key factor in patient prognosis and treatment planning while the clinical data of soft tissue sarcoma are imbalanced. In this paper, we propose an effective solution to find the optimal imbalance machine learning model for predicting the classification of soft tissue sarcoma data.MethodsIn this paper, a large number of features are first obtained based on T_1WI images using the radiomics methods.Then, we explore the methods of feature selection, sampling and classification, get 17 imbalance machine learning models based on the above features and performed extensive experiments to classify imbalanced soft tissue sarcoma data. Meanwhile, we used another dataset splitting method as well, which could improve the classification performance and verify the validity of the models.ResultsThe experimental results show that the combination of extremely randomized trees (ERT) classification algorithm using SMOTETomek and the recursive feature elimination technique (RFE) performs best compared to other methods. The accuracy of RFE+STT+ERT is 81.57% , which is close to the accuracy of biopsy, and the accuracy is 95.69% when using another dataset splitting method.ConclusionPreoperative predicting pathological grade of soft tissue sarcoma in an accurate and noninvasive manner is essential. Our proposed machine learning method (RFE+STT+ERT) can make a positive contribution to solving the imbalanced data classification problem, which can favorably support the development of personalized treatment plans for soft tissue sarcoma patients.

Construction of Digital Marketing Recommendation Model Based on Random Forest Algorithm

The traditional marketing model can no longer meet the needs of users and can not add more benefits to the enterprise, and digital marketing came into being. At present, most of the marketing focus of various enterprises is still mainly on products, and the reflection arc to market changes is long. Therefore, the formulation of marketing activities should always pay attention to changes in user needs and combine corresponding activity planning, product planning, brand building, etc., according to Changes in the target market adjust the content of marketing activities and products in real-time and, at the same time, pay attention to user feedback on products in order to iteratively update products in time, improve product market competitiveness, and optimize the user experience. In this paper, through the study and research of the traditional random forest method and some data processing algorithms, the feature selection and class imbalance problems of random forest are improved, respectively. Through the study of feature selection methods, we can maintain a balance between feature strength and relevance during feature selection and improve the final model classification effect. And through the research and experiment of the imbalanced data classification problem and the random forest algorithm, the method of the random forest model to deal with the imbalanced problem has been improved. After experimental calculation and analysis, it is found that for the effect of the minimum number of samples required for node splitting with different numbers, the best results are obtained when 2 samples are taken as the minimum number of samples required for node splitting, and the average value of the F1 evaluation is 0.1038; for different specifications, the effect of the random forest is the best using the Gini index, and the average value of its F1 evaluation is 0.1033; for the effect analysis of random forests with different numbers of trees, 7 to 10 decision trees are the best, and the F1 evaluation is the best. The average is 0.10175.

Virtual Screening of Drug Proteins Based on the Prediction Classification Model of Imbalanced Data Mining

We propose a virtual screening method based on imbalanced data mining in this paper, which combines virtual screening techniques with imbalanced data classification methods to improve the traditional virtual screening process. First, in the actual virtual screening process, we apply k-means and smote heuristic oversampling method to deal with imbalanced data. Meanwhile, to enhance the accuracy of the virtual screening process, a particle swarm optimization algorithm is introduced to optimize the parameters of the support vector machine classifier, and the concept of ensemble learning is brought in. The classification technique based on particle swarm optimization, support vector machine and adaptive boosting is used to screen the molecular docking conformation to improve the accuracy of the prediction. Finally, in the experimental construction and analysis section, the proposed method was validated using relevant data from the protein data bank database and PubChem database. The experimental results indicated that the proposed method can effectively improve the accuracy of virus screening and has practical guidance for new drug development. This research regards virtual screening as a problem of imbalanced data classification, which has obvious guiding significance and also provides a certain reference for the problems faced by virtual screening technology.

A novel Random Forest integrated model for imbalanced data classification problem

In recent years, most researchers focused on the classification problems of imbalanced data sets, and these problems are widely distributed in industrial production and medical research fields. For these highly imbalanced data sets, the ensemble method based on over-sampling is one of the most competitive techniques in the present research. However, the incorrect sampling strategy easily affected the model performance, which increased the training complexity and caused an over-fitting problem. This article proposed an equilibrium ensemble method (DCI-ISSA) with two novel techniques to conquer these shortcomings. Firstly, this paper raised an over-sampling approach (Data Center Interpolation DCI) to offer a counterbalanced data set for the single learner, which can prevent the base learners from the impact of class imbalance. Additionally, we provided a parameter optimization method for Random Forest (RF), which used the Improved Sparrow Search Algorithm (ISSA) to find the optimal parameters for different imbalanced data sets dynamically. These parameters can improve the classification performance of base classifiers and adjust to all kinds of lopsided data sets with distinct sizes. Experimental results showed that the DCI-ISSA-RF model outperforms other famous approaches for the imbalanced data sets with various dimensions.

Improved cost-sensitive representation of data for solving the imbalanced big data classification problem

Dimension reduction is a preprocessing step in machine learning for eliminating undesirable features and increasing learning accuracy. In order to reduce the redundant features, there are data representation methods, each of which has its own advantages. On the other hand, big data with imbalanced classes is one of the most important issues in pattern recognition and machine learning. In this paper, a method is proposed in the form of a cost-sensitive optimization problem which implements the process of selecting and extracting the features simultaneously. The feature extraction phase is based on reducing error and maintaining geometric relationships between data by solving a manifold learning optimization problem. In the feature selection phase, the cost-sensitive optimization problem is adopted based on minimizing the upper limit of the generalization error. Finally, the optimization problem which is constituted from the above two problems is solved by adding a cost-sensitive term to create a balance between classes without manipulating the data. To evaluate the results of the feature reduction, the multi-class linear SVM classifier is used on the reduced data. The proposed method is compared with some other approaches on 21 datasets from the UCI learning repository, microarrays and high-dimensional datasets, as well as imbalanced datasets from the KEEL repository. The results indicate the significant efficiency of the proposed method compared to some similar approaches.

Cost-Sensitive Variational Autoencoding Classifier for Imbalanced Data Classification

Classification is among the core tasks in machine learning. Existing classification algorithms are typically based on the assumption of at least roughly balanced data classes. When performing tasks involving imbalanced data, such classifiers ignore the minority data in consideration of the overall accuracy. The performance of traditional classification algorithms based on the assumption of balanced data distribution is insufficient because the minority-class samples are often more important than others, such as positive samples, in disease diagnosis. In this study, we propose a cost-sensitive variational autoencoding classifier that combines data-level and algorithm-level methods to solve the problem of imbalanced data classification. Cost-sensitive factors are introduced to assign a high cost to the misclassification of minority data, which biases the classifier toward minority data. We also designed misclassification costs closely related to tasks by embedding domain knowledge. Experimental results show that the proposed method performed the classification of bulk amorphous materials well.

A resistance outlier sampling algorithm for imbalanced data prediction

Classification of imbalanced data is an important challenge in current research. Sampling is an important way to solve the problem of imbalanced data classification, but some traditional sampling algorithms are susceptible to outliers. Therefore, an iF-ADASYN sampling algorithm is proposed in this paper. First, based on the ADASYN algorithm, we introduce the isolation Forest algorithm to overcome its vulnerability to outliers. Then, a calculation method of anomaly index which can delete outliers accurately of minority data is presented. The experimental results of four UCI public imbalanced datasets show that the algorithm can effectively improve the accuracy of the minority class, and increase the stability. In the real thrombus dataset, the AUC value of the iF-ADASYN algorithm is more significant than that of SMOTE and ADASYN algorithms, and the recognition rate of patients with thrombosis increased by 20%. The iF-ADASYN algorithm obtains better resistance to outliers than the original ADASYN algorithm. Meanwhile, it improves the accuracy of minority class decision boundary region division.

Single-Point Crossover and Jellyfish Optimization for Handling Imbalanced Data Classification Problem

The imbalanced datasets and their classification has pulled in as a hot research topic over the years. It is used in different fields, for example, security, finance, health, and many others. The imbalanced datasets are balanced by applying resampling and various solutions are designed to tackle such datasets that mainly focus on class distribution issues. The imbalanced data is rebalanced using these methods. This paper introduces a technique for balancing data through two stages: first, oversampling methods are utilized in the process of rebalancing such imbalanced dataset using the single-point crossover to generate the new data of minority classes, second, it searches for an optimal subset of the imbalanced and balanced datasets by Jellyfish Search (JS) which is an optimization method. Experiments are performed on 18 real imbalanced datasets, and results are compared with famous oversampling methods and the recently published ACOR (Ant Colony Optimization Resampling) method in terms of different appraisal measurements. Higher performance is recorded by the proposed method and comparability with well-known and recent techniques.

A Statistical Approach to Cost-Sensitive AdaBoost for Imbalanced Data Classification

To address the imbalanced data problem in classification, the studies of the combination of AdaBoost, short for “Adaptive Boosting,” and cost-sensitive learning have shown convincing results in the literature. The cost-sensitive AdaBoost algorithms are practical since the “boosting” property in AdaBoost can iteratively enhance the small class of the cost-sensitive learning to solve the imbalanced data issue. However, the most available cost-sensitive AdaBoost algorithms are heuristic approaches, which are improved from the standard AdaBoost algorithm by cost-sensitively adjusting the voting weight parameters of weak classifiers or the sample updating weight parameters without strict theoretic proof. The algorithms are appended the cost-sensitive factors to focus on the high-cost and small-class samples, but they have no procedures to show the best place to add the cost factors and the cost factor value set. To complete the cost-sensitive AdaBoost algorithms’ framework, the present article has two main contributions. First, we summarize the popular cost-sensitive boosting algorithms in the literature and propose a generally comprehensive form. We name our specific one, the “AdaImC algorithm,” which is typically appliable to solve the imbalanced data classification problem with theoretic proof. Second, a statistical approach to prove the AdaImC algorithm is proposed to verify the inner relationship between the cost parameters. We show that our proposed algorithm in the machine learning field is identical to the Product of Experts (PoE) model in the statistics field. Besides, a way to determine the cost parameter value by the statistical analysis is introduced. Several numeric studies are listed finally to support our proposed algorithm.