Supervised Discretization Methods Research Articles

Supervised discretization of continuous-valued attributes for classification using RACER algorithm

In the contemporary world, data pervades every facet of human life, and the information contained in this data plays a pivotal role in shaping decision-making and advancing technology. Among the plethora of techniques available, classification methods are highly effective tools for extracting valuable insights from vast volumes of data. The Rule Aggregation ClassifiER (RACER) is a novel rule-based classification algorithm known for its exceptional performance. A notable limitation of RACER lies in its inability to handle continuous features. In this paper, we address the aforementioned limitation by employing various supervised discretization methods, including CAIM, MDLP, Decision Tree (CART), and ChiMerge. The impact of these methods on RACER’s accuracy and understandability is evaluated across nine datasets from the UCI repository. Additionally, the paper conducts a comparative analysis of RACER’s accuracy against well-known classifiers such as Naive Bayes, Logistic Regression, SVM, LightGBM, and Decision Tree. The findings indicate that RACER achieves the highest average accuracy when we utilize MDLP as the discretization method, surpassing its initial average accuracy. Moreover, RACER demonstrates superior understandability by generating the lowest number of rules when employing ChiMerge and Decision Tree for discretizing numerical features. Furthermore, RACER outperforms the other five classifiers when employing MDLP.

LR-SDiscr: a novel and scalable merging and splitting discretization framework using a lexical generator

ABSTRACTIn this paper, we propose a novel supervised discretization method namely LR-SDiscr. It is based on a Left to Right (LR) scanning technique, which partitions automatically the input stream into intervals. Its originality resides in the fact it handles both merging and division operations in the same process, and hence it benefits of the use of a large spectrum of statistic measures. The second strength of our proposal is the reduction of the discretization complexity by processing the data in only one pass. Extensive experiments were conducted on various cut-point functions using public benchmarks that include small, large and medical datasets. LR-SDiscr outperforms several classical and recent discretization methods.

A comparison of methods for discretizing continuous variables in Bayesian Networks

Bayesian Networks (BNs) are an increasingly popular method for modelling environmental systems. The discretization of continuous variables is often required to use BNs. There are three main methods of discretization; manual, unsupervised, and supervised. Here, we compare and demonstrate each approach with a BN that predicts coastal erosion. Results reveal that supervised discretization methods produced BNs of the highest average predictive skill (73.8%), followed by manual discretization (69.0%) and unsupervised discretization (64.8%). However, each method has specific advantages that may make them more suitable for particular applications. Manual methods can produce physical meaningful BNs, which is favorable in environmental modelling. Supervised methods can autonomously and optimally discretize variables and may be preferred when predictive skill is a modelling priority. Unsupervised methods are computationally simple and versatile. The optimal discretization scheme should consider both the performance and practicality of the scheme.

Finding optimum width of discretization for gene expressions using functional annotations

Discretizing gene expression values is an important step in data preprocessing as it helps in reducing noise and experimental errors. This in turn provides better results in various tasks such as gene regulatory network analysis and disease prediction. A supervised discretization method for gene expressions using gene annotation is developed. The method is called “Gene Annotation Based Discretization” (GABD) where the discretization width is determined by maximizing the positive predictive value (PPV), computed using gene annotations, for top 20,000 gene pairs. The method can capture the gene similarity better than those obtained using original expressions. The performance of GABD is compared with some existing discretization methods like equal width discretization, equal frequency discretization and k-means discretization in terms of positive predictive value (PPV). The utility of GABD is also shown by clustering genes using k-medoid algorithm and thereby predicting the function of 23 unclassified Saccharomyces cerevisiae genes using p-value cut off 10−10. The source code for GABD is available at http://www.sampa.droppages.com/GABD.html.

A Hybrid Wind Speed Forecasting System Based on a ‘Decomposition and Ensemble’ Strategy and Fuzzy Time Series

Accurate and stable wind speed forecasting is of critical importance in the wind power industry and has measurable influence on power-system management and the stability of market economics. However, most traditional wind speed forecasting models require a large amount of historical data and face restrictions due to assumptions, such as normality postulates. Additionally, any data volatility leads to increased forecasting instability. Therefore, in this paper, a hybrid forecasting system, which combines the ‘decomposition and ensemble’ strategy and fuzzy time series forecasting algorithm, is proposed that comprises two modules—data pre-processing and forecasting. Moreover, the statistical model, artificial neural network, and Support Vector Regression model are employed to compare with the proposed hybrid system, which is proven to be very effective in forecasting wind speed data affected by noise and instability. The results of these comparisons demonstrate that the hybrid forecasting system can improve the forecasting accuracy and stability significantly, and supervised discretization methods outperform the unsupervised methods for fuzzy time series in most cases.

Comparison of the effect of unsupervised and supervised discretization methods on classification process

Most of the machine learning and data mining algorithms use discrete data for the classification process. But, most data in practice include continuous features. Therefore, a discretization pre-processing step is applied on these datasets before the classification. Discretization process converts continuous values to discrete values. In the literature, there are many methods used for discretization process. These methods are grouped as supervised and unsupervised methods according to whether a class information is used or not. In this paper, we used two unsupervised methods: Equal Width Interval (EW), Equal Frequency (EF) and one supervised method: Entropy Based (EB) discretization. In the experiments, a well-known 10 dataset from UCI (Machine Learning Repository) is used in order to compare the effect of the discretization methods on the classification. The results show that, Naive Bayes (NB), C4.5 and ID3 classification algorithms obtain higher accuracy with EB discretization method.

Comparison of the effect of unsupervised and supervised discretization methods on classification process

Most of the machine learning and data mining algorithms use discrete data for the classification process. But, most data in practice include continuous features. Therefore, a discretization pre-processing step is applied on these datasets before the classification. Discretization process converts continuous values to discrete values. In the literature, there are many methods used for discretization process. These methods are grouped as supervised and unsupervised methods according to whether a class information is used or not. In this paper, we used two unsupervised methods: Equal Width Interval (EW), Equal Frequency (EF) and one supervised method: Entropy Based (EB) discretization. In the experiments, a well-known 10 dataset from UCI (Machine Learning Repository) is used in order to compare the effect of the discretization methods on the classification. The results show that, Naive Bayes (NB), C4.5 and ID3 classification algorithms obtain higher accuracy with EB discretization method.

Supervised discretization can discover risk groups in cancer survival analysis

Discretization of continuous variables is a common practice in medical research to identify risk patient groups. This work compares the performance of gold-standard categorization procedures (TNM+A protocol) with that of three supervised discretization methods from Machine Learning (CAIM, ChiM and DTree) in the stratification of patients with breast cancer. The performance for the discretization algorithms was evaluated based on the results obtained after applying standard survival analysis procedures such as Kaplan–Meier curves, Cox regression and predictive modelling. The results show that the application of alternative discretization algorithms could lead the clinicians to get valuable information for the diagnosis and outcome of the disease. Patient data were collected from the Medical Oncology Service of the Hospital Clínico Universitario (Málaga, Spain) considering a follow up period from 1982 to 2008.

A DISCRETIZATION METHOD BASED ON MAXIMIZING THE AREA UNDER RECEIVER OPERATING CHARACTERISTIC CURVE

Many machine learning algorithms require the features to be categorical. Hence, they require all numeric-valued data to be discretized into intervals. In this paper, we present a new discretization method based on the receiver operating characteristics (ROC) Curve (AUC) measure. Maximum area under ROC curve-based discretization (MAD) is a global, static and supervised discretization method. MAD uses the sorted order of the continuous values of a feature and discretizes the feature in such a way that the AUC based on that feature is to be maximized. The proposed method is compared with alternative discretization methods such as ChiMerge, Entropy-Minimum Description Length Principle (MDLP), Fixed Frequency Discretization (FFD), and Proportional Discretization (PD). FFD and PD have been recently proposed and are designed for Naïve Bayes learning. ChiMerge is a merging discretization method as the MAD method. Evaluations are performed in terms of M-Measure, an AUC-based metric for multi-class classification, and accuracy values obtained from Naïve Bayes and Aggregating One-Dependence Estimators (AODE) algorithms by using real-world datasets. Empirical results show that MAD is a strong candidate to be a good alternative to other discretization methods.

A UMDA-Based Discretization Method for Continuous Attributes

Discretization of continuous attributes have played an important role in machine learning and data mining. They can not only improve the performance of the classifier, but also reduce the space of the storage. Univariate Marginal Distribution Algorithm is a modified Evolutionary Algorithms, which has some advantages over classical Evolutionary Algorithms such as the fast convergence speed and few parameters need to be tuned. In this paper, we proposed a bottom-up, global, dynamic, and supervised discretization method on the basis of Univariate Marginal Distribution Algorithm.The experimental results showed that the proposed method could effectively improve the accuracy of classifier.

IDD: A Supervised Interval Distance-Based Method for Discretization

This article introduces a new method for supervised discretization based on interval distances by using a novel concept of neighbourhood in the target's space. The method proposed takes into consideration the order of the class attribute, when this exists, so that it can be used with ordinal discrete classes as well as continuous classes, in the case of regression problems. The method has proved to be very efficient in terms of accuracy and faster than the most commonly supervised discretization methods used in the literature. It is illustrated through several examples and a comparison with other standard discretization methods is performed for three public data sets by using two different learning tasks: a decision tree algorithm and SVM for regression.

Optimal bin number for equal frequency discretizations in supervized learning

While real data often comes in mixed format, discrete and continuous, many supervised induction algorithms require discrete data. Although efficient supervised discretization methods are available, the unsupervised Equal Frequency discretization method is still widely used by the statistician both for data exploration and data preparation. In this paper, we propose an automatic method, based on a Bayesian approach, to optimize the number of bins for Equal Frequency discretizations in the context of supervised learning. We introduce a space of Equal Frequency discretization models and a prior distribution defined on this model space. This results in the definition of a Bayes optimal evaluation criterion for Equal Frequency discretizations. We then propose an optimal search algorithm whose run-time is super-linear in the sample size. Extensive comparative experiments demonstrate that the method works quite well in many cases.