Tomek Links Research Articles

PSU: Particle Stacking Undersampling Method for Highly Imbalanced Big Data

Imbalanced classes are a common problem in machine learning, and the computational costs required for proper resampling increases with the data size. In this study, a simple and effective undersampling method, named particle stacking undersampling (PSU) was proposed. Compared with other competing undersampling methods, PSU can significantly reduce the computational costs, while minimizing information loss to prevent a prediction bias. The performance benchmark applied on 55 binary classification problems indicated that the proposed method not only achieved an enhanced classification performance over other well-known undersampling methods (random undersampling, NearMiss-1, NearMiss-2, cluster centroid, edited nearest neighbor, condensed nearest neighbor, and Tomek Links) but also provided a computational simplicity that can be scalable to large data. Moreover, an experiment verified that two propositions forming the basis of the PSU algorithm can also be applied to other undersampling methods to achieve methodological improvements.

MLTL: A multi-label approach for the Tomek Link undersampling algorithm

A large variety of problems are multi-labeled, which made the Multi-Label Classification field become an active topic in the machine learning community. However, real world problems tend to be imbalanced, meaning that some classes may have more samples than others. Learning from imbalanced datasets is a challenging task and for that has attracted the attention of researchers that have proposed some resampling algorithms to address this problem. This work presents two main contributions: A new resampling algorithm for multi-label classification problems named MLTL - Multi-Label Tomek Link, which is based on the standard Tomek Link resampling algorithm; A multi-label imbalanceness API for the Mulan framework. Results in seven well-known datasets showed that MLTL is a competitive technique when compared to other multi-label resampling methods from the literature.

Tackling Class Imbalance Problem in Software Defect Prediction Through Cluster-Based Over-Sampling With Filtering

In practice, Software Defect Prediction (SDP) models often suffer from highly imbalanced data, which makes classifiers difficult to identify defective instances. Recently, many techniques were proposed to tackle this problem, over-sampling technique is one of the most well-known methods to address class imbalance problem. This technique balances the number of defective and non-defective instances by generating new defective instances. However, these approaches would generate non-diverse synthetic instances, and many unnecessary noise instances at the same time. Motived by this, we propose a Cluster-based Over-sampling with noise filtering (KMFOS) approach to tackle class imbalance problem in SDP. KMFOS firstly divides defective instances into $K$ clusters, and new defective instances are generated by interpolation between instances of each two clusters. After this, these new defective instances would diversely spread in the space of defective dataset. Then, we extend this cluster-based over-sampling through the Closest List Noise Identification (CLNI) to clean the noise instances. We do extensive experiments on 24 projects to compare KMFOS with some over-sampling approaches such as SMOTE, Borderline-SMOTE, ADASYN, random over-sampling (ROS), K-means SMOTE, SMOTE + IPF, SMOTE + ENN and SMOTE + Tomek Links using five prediction classifiers. At the same time, we also compare KMFOS with other state-of-the-art class-imbalance methods including balancebaggingclassifier, RUSboostclassifier, InstanceHardnessThreshold and cost-sensitive methods. Experimental results indicate our KMFOS can obtain better Recall and bal values than other over-sampling methods and other compared class-imbalance methods. Hence, KMFOS is an efficient approach to generate balanced data for SDP and improves the performance of predicting models.

SMOTETomek-Based Resampling for Personality Recognition

The main challenge of user personality recognition is low accuracy resulting from small sample size and severe sample distribution imbalance. This paper analyzes the impact of imbalanced data distribution and positive and negative sample overlap on the machine learning classification model. The classification model is based on the data resampling technique, which can improve the classification accuracy. These problems can be solved once the data are effectively resampled. We present a personality prediction method based on particle swarm optimization (PSO) and synthetic minority oversampling technique+Tomek Link (SMOTETomek)resampling (PSO-SMOTETomek), which, apart from effective SMOTETomek resampling of data samples, is able to execute PSO feature optimization for each set of feature combinations. Validated by simulation, our analysis reveals that the PSO-SMOTETomek method is efficient under a small dataset, and the accuracy of personality recognition is improved by up to around 10%. The results are better than those of previous similar studies. The average accuracies of the plain text dataset and the non-plain text dataset are 75.34% and 78.78%, respectively. The average accuracies of the short text dataset and the long text dataset are 75.34% and 64.25%, respectively. From the experimental results, we found that short text has a better classification effect than long text. Plain text data can still have high personality discrimination accuracy, but there is no relevant external information. The proposed model is able to facilitate the design and implementation of a personality recognition system, and the model significantly outperforms existing state-of-the-art models.

Development of Disease Prediction Model Based on Ensemble Learning Approach for Diabetes and Hypertension

Early diseases prediction plays an important role for improving healthcare quality and can help individuals avoid dangerous health situations before it is too late. This paper proposes a disease prediction model (DPM) to provide an early prediction for type 2 diabetes and hypertension based on individual's risk factors data. The proposed DPM consists of isolation forest (iForest) based outlier detection method to remove outlier data, synthetic minority oversampling technique tomek link (SMOTETomek) to balance data distribution, and ensemble approach to predict the diseases. Four datasets were utilized to build the model and extract the most significant risks factors. The results showed that the proposed DPM achieved highest accuracy when compared to other models and previous studies. We also developed a mobile application to provide the practical application of the proposed DPM. The developed mobile application gathers risk factor data and send it to a remote server, so that an individual's current condition can be diagnosed with the proposed DPM. The prediction result is then sent back to the mobile application; thus, immediate and appropriate action can be taken to reduce and prevent individual's risks once unexpected health situations occur (i.e., type 2 diabetes and/or hypertension) at early stages.

Bagging of Xgboost Classifiers with Random Under-sampling and Tomek Link for Noisy Label-imbalanced Data

Fitting label-imbalanced data with high level of noise is one of the major challenges in learning-based intelligent system design. In this paper, for the two-class problem, we propose a bagging-based algorithm with Xgboost classifier (Gradient Boosting Machine) and under-sampling approaches to overcome the challenge. To avoid model misspecification caused by imbalanced data, random sampling with replacement is employed to obtain several balanced training sets; and to mitigate the problem of misleading information produced by noise, Tomek Link method is introduced to eliminate the cross-class overlapped instances, which are the primal sources of noise. And to obtain robust individual learners, we utilize Xgboost, a novel Gradient Boosting Machine-based classifier with convenient parameter tuning interface, to fit each component of the bagging ensemble. The performance of the proposed method is tested with Mandarin radio records (MFCC features) with the task of keywords recognition, and experimental results show that the new method could outperform single Xgboost classifier, verified the rationality and effectiveness of the bagging scheme. The method proposed in the paper could offer a novel solution to the challenge of noisy imbalanced data classification, and the implementation of Xgboost in this area could also serve as an innovative work.

Machine learning for automatic rule classification of agricultural regulations: A case study in Spain

Currently, pest management practices require modern equipment and the use of complex information, such as regulations and guidelines. The complexity of regulations is the root cause of the emergence of automated solutions for compliance assessment by translating regulations into sets of machine-processable rules that can be run by specialized modules of farm management information systems (FMIS). However, the manual translation of rules is prohibitively costly, and therefore, this translation should be carried out with the support of artificial intelligence techniques.In this paper, we use the official Spanish phytosanitary products registry to empirically evaluate the performance of four popular machine learning algorithms in the task of correctly classifying pesticide regulations as prohibitions or obligations. Moreover, we also evaluate how to improve the performance of the algorithms in the preprocessing of the texts with natural language processing techniques. Finally, due to the specific characteristics of the texts found in pesticide regulations, resampling techniques are also evaluated. Experiments show that the combination of the machine learning algorithm Logic regression, the natural language technique part-of-speech tagging and the resampling technique Tomek links is the best performing approach, with an F1 score of 68.8%, a precision of 84.46% and a recall of 60%. The experimental results are promising, and they show that this approach can be applied to develop a computer-aided tool for transforming textual pesticide regulations into machine-processable rules. To the best of our knowledge, this is the first study that evaluates the use of artificial intelligence methods for the automatic translation of agricultural regulations into machine-processable representations.

Classification of Imbalance Data using Tomek Link(T-Link) Combined with Random Under-sampling (RUS) as a Data Reduction Method

Classification of Imbalance Data using Tomek Link(T-Link) Combined with Random Under-sampling (RUS) as a Data Reduction Method

Classification of Imbalance Data using Tomek Link (T-Link) Combined with Random Under-sampling (RUS) as a Data Reduction Method

The problem of classifying subjects into disease categories is of common occurrence in medical research. Machine learning tools such as Artificial Neural Network (ANN), Support Vector Machine (SVM) and Logistic Regression (LR) and Fisher’s Linear Discriminant Analysis (LDA) are widely used in the areas of prediction and classification. The main objective of these competing classification strategies is to predict a dichotomous outcome (e.g. disease/healthy) based on several features. Like any of the well-known statistical inferential models; machine learning tools are faced with a problem known as “class imbalance”. A data set is imbalanced if the classification categories are not approximately equally represented. When learning from highly imbalanced data, most classifiers are affected by the majority class leading to an increase in the false negative rate. Increased interests in applying machine learning techniques to real-world problems, whose data are characterized by severe imbalance, have emerged as can be seen in numerous publications in medicine and biology. Predictive accuracy, a popular choice for evaluating performance of a classifier, might not be appropriate when the data is imbalanced and/or when the costs of different errors vary markedly. In this paper, we use the T-Link algorithm in the preprocessing phase as a method of data cleaning in order to remove noise. We combine T-Link with other sampling method such as RUS, ROS and Synthetic Minority Technique (SMOTE) in order to maintain a balanced class distribution. Classification was then utilized using several ML algorithms such as ANN, RF and LR. Classifiers performance was evaluated using several performance measures deemed more appropriate for classifying data with sever imbalance. These methods are applied to arterial blood pressures data and Ecoli2 data set. Using TLink in combination with RUS and SMOTE demonstrated a superior performance compared to resampling techniques such among different classification algorithms such as SVM, ANN, RF and LR.

Combine Sampling Support Vector Machine for Imbalanced Data Classification

Imbalanced data are defined as dataset condition with some class is larger than any class in number: the larger class described as the majority or negative class and the less class described as minority or positive class. This condition considerate as problem in data classification since most of classifiers tend to predict major class and ignore minor class, hence the analysis provide lack of accuracy for minor class. Some basic ideas on the approach to the data level by using sampling-based approaches to handle this classification issue are under sampling and oversampling. Synthetic minority oversampling technique (SMOTE) is one of oversampling methods to increase number of positive class using sample drawing techniques by randomly replicate the data in such way that the number of positive class is equal to the number of negative class. Other method is Tomek links, an under sampling method and works by decreasing the number of negative class. In this research, combine sampling was done by combining SMOTE and Tomek links techniques along with SVM as the binary classification method. Based on accuracy rates in this study, using combine sampling method provided better result than SMOTE and Tomek links in 5-fold cross validation. However, in some extreme cases combine sampling method are no better than the use of methods Tomek links.

Using pre & post-processing methods to improve binding site predictions

Currently the best algorithms for transcription factor binding site prediction within sequences of regulatory DNA are severely limited in accuracy. In this paper, we integrate 12 original binding site prediction algorithms, and use a ‘window’ of consecutive predictions in order to contextualise the neighbouring results. We combine either random selection or Tomek links under-sampling with SMOTE over-sampling techniques. In addition, we investigate the behaviour of four feature selection filtering methods: bi-normal separation, correlation coefficients, F-Score and a cross entropy based algorithm. Finally, we remove some of the final predicted binding sites on the basis of their biological plausibility. The results show that we can generate a new prediction that significantly improves on the performance of any one of the individual algorithms.

Bayesian Induction of Verb Sub-categorization Frames in Imbalanced Heterogeneous Data

The problem of high imbalance in data in the binary classification task of determining whether a syntactic construction (environment) co-occurring with a verb in a natural text corpus consists of a subcategorization frame of the verb or not is the central focus of the present paper. Each environment is encoded as a vector of heterogeneous attributes, where a very high imbalance between positive and negative examples is observed (an imbalance ratio of approximately 1:80). In order to cope with the plethora of negative examples, we propose a search tactic during training that employs Tomek links for eliminating unnecessary negative examples from the training set. As for a classification mechanism, we argue that Bayesian networks are well suited and we propose a novel network structure which efficiently handles heterogeneous attributes without discretization and is more classification-oriented. Comparing the experimental results with those of other known machine learning algorithms, our methodology performs significantly better in detecting instances of the rare positive class.

Automated design of linear tree classifiers

An automated method is presented for the design of linear tree classifiers, i.e. tree classifiers in which a decision based on a linear sum of features is carried out at each node. The method exploits the discriminability of Tomek links joining opposed pairs of data points in multidimensional feature space to produce a hierarchically structured piecewise linear decision function. The corresponding decision surface is optimized by a gradient descent that maximizes the number of Tomek links cut by each linear segment of the decision surface, followed by training each node's linear decision segment on the data associated with that node. Experiments on real data obtained from ship images and character images suggest that the accuracy of the tree classifier designed by this scheme is comparable to that of the k-NN classifier while providing much greater decision speeds, and that the trade-off between the speed and the accuracy in pattern classification can be controlled by bounding the number of features to be used at each node of the tree. Further experiments comparing the classification errors of our tree classifier with the tree classifier produced by the Mui/Fu technique indicate that our tree classifier is no less accurate and often much faster than the Mui/Fu classifier.