Cluster-based Undersampling Research Articles

Glypred: Lysine Glycation Site Prediction via CCU-LightGBM-BiLSTM Framework with Multi-Head Attention Mechanism.

Glycation, a type of posttranslational modification, preferentially occurs on lysine and arginine residues, impairing protein functionality and altering characteristics. This process is linked to diseases such as Alzheimer's, diabetes, and atherosclerosis. Traditional wet lab experiments are time-consuming, whereas machine learning has significantly streamlined the prediction of protein glycation sites. Despite promising results, challenges remain, including data imbalance, feature redundancy, and suboptimal classifier performance. This research introduces Glypred, a lysine glycation site prediction model combining ClusterCentroids Undersampling (CCU), LightGBM, and bidirectional long short-term memory network (BiLSTM) methodologies, with an additional multihead attention mechanism integrated into the BiLSTM. To achieve this, the study undertakes several key steps: selecting diverse feature types to capture comprehensive protein information, employing a cluster-based undersampling strategy to balance the data set, using LightGBM for feature selection to enhance model performance, and implementing a bidirectional LSTM network for accurate classification. Together, these approaches ensure that Glypred effectively identifies glycation sites with high accuracy and robustness. For feature encoding, five distinct feature types─AAC, KMER, DR, PWAA, and EBGW─were selected to capture a broad spectrum of protein sequence and biological information. These encoded features were integrated and validated to ensure comprehensive protein information acquisition. To address the issue of highly imbalanced positive and negative samples, various undersampling algorithms, including random undersampling, NearMiss, edited nearest neighbor rule, and CCU, were evaluated. CCU was ultimately chosen to remove redundant nonglycated training data, establishing a balanced data set that enhances the model's accuracy and robustness. For feature selection, the LightGBM ensemble learning algorithm was employed to reduce feature dimensionality by identifying the most significant features. This approach accelerates model training, enhances generalization capabilities, and ensures good transferability of the model. Finally, a bidirectional long short-term memory network was used as the classifier, with a network structure designed to capture glycation modification site features from both forward and backward directions. To prevent overfitting, appropriate regularization parameters and dropout rates were introduced, achieving efficient classification. Experimental results show that Glypred achieved optimal performance. This model provides new insights for bioinformatics and encourages the application of similar strategies in other fields. A lysine glycation site prediction software tool was also developed using the PyQt5 library, offering researchers an auxiliary screening tool to reduce workload and improve efficiency. The software and data sets are available on GitHub: https://github.com/ZBYnb/Glypred.

Highly Imbalanced Railway Station Structural Damage Monitoring Based on Cluster-Based Undersampling and Siamese Artificial Neural Network

Highly Imbalanced Railway Station Structural Damage Monitoring Based on Cluster-Based Undersampling and Siamese Artificial Neural Network

New boosting approaches for improving cluster-based undersampling in problems with imbalanced data

Class unbalanced datasets are frequently encountered in a variety of areas including health, security, and finance. Often these datasets create bias in the supervised learning models trained for the prediction task. One of the most successful techniques to handle imbalanced data is undersampling. Experiments demonstrate that cluster-based undersampling improves over random undersampling in many cases. In this paper, we propose three new boosting approaches to improve the performance of cluster-based undersampling technique: (i) inject unlabeled data into training data for improved clustering; (ii) keep the instances close to cluster boundary and centroid while undersampling and (iii) remove the majority samples in the neighborhood of minority data in each cluster. We experimented with our boosting methods over 49 standard benchmark datasets and analyzed the performances in terms of standard evaluation metrics. Experimental results suggest these boosting techniques are promising and significantly improve over cluster-based undersampling strategies.

A Cluster-based Undersampling Technique for Multiclass Skewed Datasets

Imbalanced data classification is a demanding issue in data mining and machine learning. Models that learn with imbalanced input generate feeble performance in the minority class. Resampling methods can handle this issue and balance the skewed dataset. Cluster-based Undersampling (CUS) and Near-Miss (NM) techniques are widely used in imbalanced learning. However, these methods suffer from some serious flaws. CUS averts the impact of the distance factor on instances over the majority class. Near-miss method discards the inter-class data within the majority of class elements. To overcome these flaws, this study has come up with an undersampling technique called Adaptive K-means Clustering Undersampling (AKCUS). The proposed technique blends the distance factor and clustering over the majority class. The performance of the proposed method was analyzed with the aid of an experimental study. Three multiminority datasets with different imbalance ratios were selected and the models were created using K-Nearest Neighbor (kNN), Decision Tree (DT), and Random Forest (RF) classifiers. The experimental results show that AKCUS can attain better efficacy than the benchmark methods over multiminority datasets with high imbalance ratios.

Bankruptcy prediction using synthetic sampling

The prediction of corporate bankruptcy has been widely studied. However, low bankruptcy rates lead to highly imbalanced bankruptcy class distributions, increasing the difficulty of accurately predicting a firm’s bankruptcy. Based on a large sample of 1824 U.S. firms, the study shows that classification accuracy significantly improves when the training dataset is balanced using the synthetic minority oversampling technique or one of its extensions. The results indicate that combining SMOTE with cluster-based undersampling leads to the best classification performance, and the increase in accuracy, specifically in terms of recall and AUC, is significant, thus justifying synthetic sampling when training a bankruptcy prediction classifier.

A resampling approach to disaggregate analysis of bus-involved crashes using panel data with excessive zeros

Public bus constitutes more than 70% of the overall road-based public transport patronage in Hong Kong, and its crash involvement rate has been the highest among all public transport modes. Though previous studies had identified explanatory factors that affect the crash risk of buses, use of considerably imbalanced crash data with excessive zero observations could lead to inaccurate parameter estimation. This study aims to resolve the excess zero problem of disaggregate analysis of bus-involved crashes based on synthetic data using a Synthetic Minority Over-Sampling Technique for panel data (SMOTE-P). Dataset comprising crash, traffic, and road inventory data of 88 road segments in Hong Kong during the period from 2014 to 2017 is used. To assess the data balancing performance, other common data generation approaches such as Random Under-sampling of the Majority Class (RUMC) technique, Cluster-Based Under-Sampling (CBUS), and mixed resampling, are also considered. Random effect Poisson (REP) models based on synthetic data and random effect zero-inflated Poisson (REZIP) model based on original data are estimated. Results indicate that REP model based on synthetic data using SMOTE-P outperforms REZIP model based on original data and REP models based on synthetic data using RUMC, CBUS and mixed approaches, in terms of statistical fit, prediction error, and explanatory factors identified. Results of model estimation based on SMOTE-P suggest that factors including morning peak, evening peak, hourly traffic flow, average lane width, road length, bus stop density, percentage of bus in the traffic stream, and presence of bus priority lane all affect the bus-involved crash frequency. More importantly, this study provides a feasible solution for disaggregate crash analysis with imbalanced panel data.

HCBST: An Efficient Hybrid Sampling Technique for Class Imbalance Problems

Class imbalance problem is prevalent in many real-world domains. It has become an active area of research. In binary classification problems, imbalance learning refers to learning from a dataset with a high degree of skewness to the negative class. This phenomenon causes classification algorithms to perform woefully when predicting positive classes with new examples. Data resampling, which involves manipulating the training data before applying standard classification techniques, is among the most commonly used techniques to deal with the class imbalance problem. This article presents a new hybrid sampling technique that improves the overall performance of classification algorithms for solving the class imbalance problem significantly. The proposed method called the Hybrid Cluster-Based Undersampling Technique (HCBST) uses a combination of the cluster undersampling technique to under-sample the majority instances and an oversampling technique derived from Sigma Nearest Oversampling based on Convex Combination, to oversample the minority instances to solve the class imbalance problem with a high degree of accuracy and reliability. The performance of the proposed algorithm was tested using 11 datasets from the National Aeronautics and Space Administration Metric Data Program data repository and University of California Irvine Machine Learning data repository with varying degrees of imbalance. Results were compared with classification algorithms such as the K-nearest neighbours, support vector machines, decision tree, random forest, neural network, AdaBoost, naïve Bayes, and quadratic discriminant analysis. Tests results revealed that for the same datasets, the HCBST performed better with average performances of 0.73, 0.67, and 0.35 in terms of performance measures of area under curve, geometric mean, and Matthews Correlation Coefficient, respectively, across all the classifiers used for this study. The HCBST has the potential of improving the performance of the class imbalance problem, which by extension, will improve on the various applications that rely on the concept for a solution.

The Empirical Comparison of Machine Learning Algorithm for the Class Imbalanced Problem in Conformational Epitope Prediction

A conformational epitope is a part of a protein-based vaccine. It is challenging to identify using an experiment. A computational model is developed to support identification. However, the imbalance class is one of the constraints to achieving optimal performance on the conformational epitope B cell prediction. In this paper, we compare several conformational epitope B cell prediction models from non-ensemble and ensemble approaches. A sampling method from Random undersampling, SMOTE, and cluster-based undersampling is combined with a decision tree or SVM to build a non-ensemble model. A random forest model and several variants of the bagging method is used to construct the ensemble model. A 10-fold cross-validation method is used to validate the model. The experiment results show that the combination of the cluster-based under-sampling and decision tree outperformed the other sampling method when combined with the non-ensemble and the ensemble method. This study provides a baseline to improve existing models for dealing with the class imbalance in the conformational epitope prediction.

Breast Cancer Diagnosis Using Cluster-based Undersampling and Boosted C5.0 Algorithm

Learning from imbalanced data set is relatively new challenge for breast cancer diagnosis, where the diseases cases are often quite rare relative to normal population. Although traditional algorithms are all accuracy-oriented which result biased towards the majority class. The combinations of sampling methods with ensemble classifiers have shown certainly good performance. In this paper, a hybrid of cluster-based undersampling and boosted C5.0 is proposed. The proposed classification model consists of two phases: cluster analysis and classification. In cluster analysis, affinity propagation algorithm is used to define the number of clusters, and then the k-means clustering is utilized to select the border and informative samples. In the classification phase, C5.0 algorithm is used in conjunction with boosting technical, owing to leverage the strength of the individual classifiers. The proposed algorithm is assessed by 14 benchmark imbalanced data sets taken from UCI dataset repository. The extensive experimental results on different imbalanced datasets demonstrated that the proposed algorithm can achieve better classification performance in terms of Matthews’ Correlation Coefficient (MCC) as compared to other existing imbalanced dataset classification algorithms.

A Boosting-Aided Adaptive Cluster-Based Undersampling Approach for Treatment of Class Imbalance Problem

The subject of a class imbalance is a well-investigated topic which addresses performance degradation of standard learning models due to uneven distribution of classes in a dataspace. Cluster-based undersampling is a popular solution in the domain which offers to eliminate majority class instances from a definite number of clusters to balance the training data. However, distance-based elimination of instances often got affected by the underlying data distribution. Recently, ensemble learning techniques have emerged as effective solution due to its weighted learning principle of rare instances. In this article, a boosting aided adaptive cluster-based undersampling technique is proposed to facilitate elimination of learning- insignificant majority class instances from the clusters, detected through AdaBoost ensemble learning model. The proposed work is validated with seven existing cluster based undersampling techniques for six binary datasets and three classification models. The experimental results have established the effectives of the proposed technique than the existing methods.

Enhancement of conformational B-cell epitope prediction using CluSMOTE.

BackgroundA conformational B-cell epitope is one of the main components of vaccine design. It contains separate segments in its sequence, which are spatially close in the antigen chain. The availability of Ag-Ab complex data on the Protein Data Bank allows for the development predictive methods. Several epitope prediction models also have been developed, including learning-based methods. However, the performance of the model is still not optimum. The main problem in learning-based prediction models is class imbalance.MethodsThis study proposes CluSMOTE, which is a combination of a cluster-based undersampling method and Synthetic Minority Oversampling Technique. The approach is used to generate other sample data to ensure that the dataset of the conformational epitope is balanced. The Hierarchical DBSCAN algorithm is performed to identify the cluster in the majority class. Some of the randomly selected data is taken from each cluster, considering the oversampling degree, and combined with the minority class data. The balance data is utilized as the training dataset to develop a conformational epitope prediction. Furthermore, two binary classification methods, Support Vector Machine and Decision Tree, are separately used to develop model prediction and to evaluate the performance of CluSMOTE in predicting conformational B-cell epitope. The experiment is focused on determining the best parameter for optimal CluSMOTE. Two independent datasets are used to compare the proposed prediction model with state of the art methods. The first and the second datasets represent the general protein and the glycoprotein antigens respectively.ResultThe experimental result shows that CluSMOTE Decision Tree outperformed the Support Vector Machine in terms of AUC and Gmean as performance measurements. The mean AUC of CluSMOTE Decision Tree in the Kringelum and the SEPPA 3 test sets are 0.83 and 0.766, respectively. This shows that CluSMOTE Decision Tree is better than other methods in the general protein antigen, though comparable with SEPPA 3 in the glycoprotein antigen.

Multi-Feature Approach for Bug Severity Assignment

When bug reports are submitted through bug tracking systems, they are analysed manually to identify their severity levels. A severity level specifies the negative impact of a bug on a system. With the huge number of submitted reports, setting the severity class manually is tedious and time consuming. Moreover, some bug types are reported more often than other types, which leads to imbalanced bug repositories. This paper proposes a multi-feature approach for automatic severity assignment, which leverages lexical, semantic, and categorical properties of the bug reports. The proposed approach utilizes word embeddings, topic model, vector space model, and an adapted K-Nearest Neighbour technique. Moreover, the impact of utilizing two sampling techniques, namely SMOTE and cluster-based under-sampling (CBU), were investigated. Experiments over two open source repositories, Eclipse and Mozilla, demonstrated that the proposed approach is superior to two previous studies.

Prediction of Breast Cancer from Imbalance Respect Using Cluster-Based Undersampling Method.

To overcome the two-class imbalanced problem existing in the diagnosis of breast cancer, a hybrid of K-means and Boosted C5.0 (K-Boosted C5.0) is proposed which is based on undersampling. K-means is utilized to select the informative samples near the boundary. During the training phase, the K-means algorithm clusters the majority and minority instances and selects a similar number of instances from each cluster. Boosted C5.0 is then used as the classifier. As there is one different instance selection factor via clustering that encourages the diversity of the training subspace in K-Boosted C5.0, it would be a great advantage to get better performance. To test the performance of the new hybrid classifier, it is implemented on 12 small-scale and 2 large-scale datasets, which are the often used datasets in class imbalanced learning. The extensive experimental results show that our proposed hybrid method outperforms most of the competitive algorithms in terms of Matthews' correlation coefficient (MCC) and accuracy indices. It can be a good alternative to the well-known machine learning methods.

Design of a Clinical Decision Support System for Fracture Prediction Using Imbalanced Dataset.

More than 1 billion people suffer from chronic respiratory diseases worldwide, accounting for more than 4 million deaths annually. Inhaled corticosteroid is a popular medication for treating chronic respiratory diseases. Its side effects include decreased bone mineral density and osteoporosis. The aims of this study are to investigate the association of inhaled corticosteroids and fracture and to design a clinical support system for fracture prediction. The data of patients aged 20 years and older, who had visited healthcare centers and been prescribed with inhaled corticosteroids within 2002–2010, were retrieved from the National Health Insurance Research Database (NHIRD). After excluding patients diagnosed with hip fracture or vertebrate fractures before using inhaled corticosteroid, a total of 11645 patients receiving inhaled corticosteroid therapy were included for this study. Among them, 1134 (9.7%) were diagnosed with hip fracture or vertebrate fracture. The statistical results showed that demographic information, chronic respiratory diseases and comorbidities, and corticosteroid-related variables (cumulative dose, mean exposed daily dose, follow-up duration, and exposed duration) were significantly different between fracture and nonfracture patients. The clinical decision support systems (CDSSs) were designed with integrated genetic algorithm (GA) and support vector machine (SVM) by training and validating the models with balanced training sets obtained by random and cluster-based undersampling methods and testing with the imbalanced NHIRD dataset. Two different objective functions were adopted for obtaining optimal models with best predictive performance. The predictive performance of the CDSSs exhibits a sensitivity of 69.84–77.00% and an AUC of 0.7495–0.7590. It was concluded that long-term use of inhaled corticosteroids may induce osteoporosis and exhibit higher incidence of hip or vertebrate fractures. The accumulated dose of ICS and OCS therapies should be continuously monitored, especially for patients with older age and women after menopause, to prevent from exceeding the maximum dosage.

Optimization of cluster-based evolutionary undersampling for the artificial neural networks in corporate bankruptcy prediction

We suggest an optimization approach of cluster-based undersampling to select appropriate instances. This approach can solve the data imbalance problem, which can lead to knowledge extraction for improving the performance of existing data mining techniques. Although data mining techniques among various big data analytics technologies have been successfully applied and proven in terms of classification performance in various domains, such as marketing, accounting and finance areas, the data imbalance problem has been regarded as one of the most important issues to be considered.We examined the effectiveness of a hybrid method using a clustering technique and genetic algorithms based on the artificial neural networks model to balance the proportion between the minority class and majority class. The objective of this paper is to constitute the best suitable training dataset for both decreasing data imbalance and improving the classification accuracy. We extracted the properly balanced dataset composed of optimal or near-optimal instances for the artificial neural networks model. The main contribution of the proposed method is that we extract explorative knowledge based on recognition of the data structure and categorize instances through the clustering technique while performing simultaneous optimization for the artificial neural networks modeling. In addition, we can easily understand why the instances are selected by the rule-format knowledge representation increasing the expressive power of the criteria of selecting instances. The proposed method is successfully applied to the bankruptcy prediction problem using financial data for which the proportion of small- and medium-sized bankruptcy firms in the manufacturing industry is extremely small compared to that of non-bankruptcy firms.