Entropy-based Discretization Research Articles

Reduced Data Sets and Entropy-Based Discretization

Results of experiments on numerical data sets discretized using two methods—global versions of Equal Frequency per Interval and Equal Interval Width-are presented. Globalization of both methods is based on entropy. For discretized data sets left and right reducts were computed. For each discretized data set and two data sets, based, respectively, on left and right reducts, we applied ten-fold cross validation using the C4.5 decision tree generation system. Our main objective was to compare the quality of all three types of data sets in terms of an error rate. Additionally, we compared complexity of generated decision trees. We show that reduction of data sets may only increase the error rate and that the decision trees generated from reduced decision sets are not simpler than the decision trees generated from non-reduced data sets.

Merging of Numerical Intervals in Entropy-Based Discretization.

As previous research indicates, a multiple-scanning methodology for discretization of numerical datasets, based on entropy, is very competitive. Discretization is a process of converting numerical values of the data records into discrete values associated with numerical intervals defined over the domains of the data records. In multiple-scanning discretization, the last step is the merging of neighboring intervals in discretized datasets as a kind of postprocessing. Our objective is to check how the error rate, measured by tenfold cross validation within the C4.5 system, is affected by such merging. We conducted experiments on 17 numerical datasets, using the same setup of multiple scanning, with three different options for merging: no merging at all, merging based on the smallest entropy, and merging based on the biggest entropy. As a result of the Friedman rank sum test (5% significance level) we concluded that the differences between all three approaches are statistically insignificant. There is no universally best approach. Then, we repeated all experiments 30 times, recording averages and standard deviations. The test of the difference between averages shows that, for a comparison of no merging with merging based on the smallest entropy, there are statistically highly significant differences (with a 1% significance level). In some cases, the smaller error rate is associated with no merging, in some cases the smaller error rate is associated with merging based on the smallest entropy. A comparison of no merging with merging based on the biggest entropy showed similar results. So, our final conclusion was that there are highly significant differences between no merging and merging, depending on the dataset. The best approach should be chosen by trying all three approaches.

Discretization Algorithm Based on Particle Swarm Optimization and Its Application in Attributes Reduction for Fault Data

In order to increase the fault diagnosis efficiency and make the fault data mining be realized, the decision table containing numerical attributes must be discretized for further calculations. The discernibility matrix-based reduction method depends on whether the numerical attributes can be properly discretized or not. So a discretization algorithm based on particle swarm optimization (PSO) is proposed. Moreover, hybrid weights are adopted in the process of particles evolution. Comparative calculations for certain equipment are completed to demonstrate the effectiveness of the proposed algorithm. The results indicate that the proposed algorithm has better performance than other popular algorithms such as class-attribute interdependence maximization (CAIM) discretization method and entropy-based discretization method.

Online entropy-based discretization for data streaming classification

Data quality is deemed as determinant in the knowledge extraction process. Low-quality data normally imply low-quality models and decisions. Discretization, as part of data preprocessing, is considered one of the most relevant techniques for improving data quality.In static discretization, output intervals are generated at once, and maintained along the whole process. However, many contemporary problems demands rapid approaches capable of self-adapting their discretization schemes to an ever-changing nature. Other major issues for stream-based discretization such as interval definition, labeling or how is implemented the interaction between learning and discretization components are also discussed in this paper.In order to address all the aforementioned problems, we propose a novel, online and self-adaptive discretization solution for streaming classification which aims at reducing the negative impact of fluctuations in evolving intervals. Experiments with a long list of standard streaming datasets and discretizers have demonstrated that our proposal performs significantly more accurately than the other alternatives. In addition, our scheme is able to leverage from class information without incurring in an overweight cost, being ranked as one of the most rapid supervised options.

Software Fault Prediction Using Filtering Feature Selection in Cluster-Based Classification

The high accuracy of software fault prediction can help testing effort and improving software quality. Previous researchers had proposed the combination of Entropy-Based Discretization (EBD) and Cluster-Based Classification (CBC). However, the irrelevant and redundant features in software fault dataset tend to decrease the prediction accuracy value. This study proposes improvement of CBC outcomes by integrating filtering feature selection methods. Filtering feature selection methods that will be integrated with CBC i.e. Information Gain (IG), Gain Ratio (GR), and One-R (OR). Based on the research using 2 datasets NASA public MDP (i.e. PC2 and PC3), the result shows that the combination of CBC and IG yields the best average accuracy value compared to GR and OR. It generates 67.52% average of probability detection (pd) and 37.42% average of probability false alarm (pf). While CBC without feature selection yields 65.38% average pd and 49.95% average pf. It can be concluded that IG can improve CBC outcomes by increasing 2.14% average pd and reducing 12.53% average pf

Perbaikan Prediksi Kesalahan Perangkat Lunak Menggunakan Seleksi Fitur dan Cluster-Based Classification

High balance value of software fault prediction can help in conducting test effort, saving test costs, saving test resources, and improving software quality. Balance values in software fault prediction need to be considered, as in most cases, the class distribution of true and false in the software fault data set tends to be unbalanced. The balance value is obtained from trade-off between probability detection (pd) and probability false alarm (pf). Previous researchers had proposed Cluster-Based Classification (CBC) method which was integrated with Entropy-Based Discretization (EBD). However, predictive models with irrelevant and redundant features in data sets can decrease balance value. This study proposes improvement of software fault prediction outcomes on CBC by integrating feature selection methods. Some feature selection methods are integrated with CBC, i.e. Information Gain (IG), Gain Ration (GR), One-R (OR), Relief-F (RFF), and Symmetric Uncertainty (SU). The result shows that combination of CBC with IG gives best average balance value, compared to other feature selection methods used in this research. Using five NASA public MDP data sets, the combination of IG and CBC generates 63.91% average of balance, while CBC method without feature selection produce 54.79% average of balance. It shows that IG can increase CBC balance average by 9.12%.

Learning to Detect Traffic Incidents from Data Based on Tree Augmented Naive Bayesian Classifiers

This study develops a tree augmented naive Bayesian (TAN) classifier based incident detection algorithm. Compared with the Bayesian networks based detection algorithms developed in the previous studies, this algorithm has less dependency on experts’ knowledge. The structure of TAN classifier for incident detection is learned from data. The discretization of continuous attributes is processed using an entropy-based method automatically. A simulation dataset on the section of the Ayer Rajah Expressway (AYE) in Singapore is used to demonstrate the development of proposed algorithm, including wavelet denoising, normalization, entropy-based discretization, and structure learning. The performance of TAN based algorithm is evaluated compared with the previous developed Bayesian network (BN) based and multilayer feed forward (MLF) neural networks based algorithms with the same AYE data. The experiment results show that the TAN based algorithms perform better than the BN classifiers and have a similar performance to the MLF based algorithm. However, TAN based algorithm would have wider vista of applications because the theory of TAN classifiers is much less complicated than MLF. It should be found from the experiment that the TAN classifier based algorithm has a significant superiority over the speed of model training and calibration compared with MLF.

Entropy-based discretization methods for ranking data

Label Ranking (LR) problems are becoming increasingly important in Machine Learning. While there has been a significant amount of work on the development of learning algorithms for LR in recent years, there are not many pre-processing methods for LR. Some methods, like Naive Bayes for LR and APRIORI-LR, cannot handle real-valued data directly. Conventional discretization methods used in classification are not suitable for LR problems, due to the different target variable. In this work, we make an extensive analysis of the existing methods using simple approaches. We also propose a new method called EDiRa (Entropy-based Discretization for Ranking) for the discretization of ranking data. We illustrate the advantages of the method using synthetic data and also on several benchmark datasets. The results clearly indicate that the discretization is performing as expected and also improves the results and efficiency of the learning algorithms.

A hybrid fuzzy time series model based on granular computing for stock price forecasting

Given the high potential benefits and impacts of accurate stock market predictions, considerable research attention has been devoted to time series forecasting for stock markets. Over long periods, the accuracy of fuzzy time series model forecasting is invariably affected by interval length, and formulating effective interval partitioning methods can be very difficult. Previous studies largely relied on distance partitioning, but this approach neglects the distribution of datasets and can only handle scalar forecasting. But the magnitude of stock price movements is often severe and difficult to predict. Thus, the distribution of stock price datasets is always skewed and the straightforward partitioning method is not well suited to these types of time series datasets. In this research, a novel fuzzy time series model is used to forecast stock market prices. The proposed model is based on the granular computing approach with binning-based partition and entropy-based discretization methods. The proposed model is verified using experimental datasets from the Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX), Dow-Jones Industrial Average (DJIA), S&P 500 and IBOVESPA stock indexes, and results are compared against existing fuzzy time series models, three different SVM models, and three modern economic models - GARCH, GJR-GARCH, and Fuzzy GARCH. Compared to other current forecasting methods, the proposed models provide improved prediction accuracy and the results are verified by paired two-tailed t-tests. The experimental results clearly provide improvements for obtaining optimized linguistic intervals and ensuring the accuracy of the proposed model.

Comparative Analysis of Variations of Ant-Miner by Varying Input Parameters

Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. ACO can be applied to the data mining field to extract rule-based classifiers. This paper presents variations of Ant-Miner named cAnt-Miner (Ant-Miner coping with continuous attributes), which incorporates an entropy-based discretization method in order to cope with continuous attributes during the rule construction process and Ant-Tree-Miner (constructing decision trees based on ACO) which generates classifications rules always in graphical form (Decision Tree). Three algorithms (Ant-Miner, Ant-Tree-Miner and cAnt-Miner) are compared against input parameters with respect to predictive accuracy and simplicity of the discovered rules.

Functional connectivity pattern extraction using entropy-based discretization and SMO

Functional connectivity pattern extraction using entropy-based discretization and SMO

A Discretization Algorithm Based on Extended Gini Criterion

A supervised, global and static algorithm for the discretization of continuous features is presented in this paper. The proposed discretization algorithm extends Gini-criterion concept. The algorithm hybridizes binning, and entropy method based on the amendment from 1R and Ent-Minimum Description Length Principle (MDLP) method. The proposed algorithm is comparable due to its simplicity, yet need further improvement and testing using various data sets. The proposed algorithm can be a good alternative method in the entropy-based discretization family.

An Alternate Method Of Examining Ipo Returns

Abstract The study proposes an alternate method, a soft computing system that mixes an entropy-based discretization method, fuzzy-rule-similarity, and rough set theory, for solving the real-world problems of initial public offerings (IPO) returns faced by both academicians and practitioners. The proposed method is illustrated by examining two practiced datasets for publicly traded firms in Taiwan. The experimental results reveal that the proposed method outperforms the listing methods in terms of accuracy and number of rules. Furthermore, the proposed method generates a set of comprehensible decision rules that can be applied in a knowledge-based system for investment decision-making of investors.

Discretization Numerical Data for Relational Data with One-to-Many Relations

Problem statement: Handling numerical data stored in a relational data base has been performed differently from handling those numerical data stored in a single table due to the multiple occurrences (one-to-many association) of an individ ual record in the non-target table and non-determin ate relations between tables. Numbers in Multi-Relation al Data Mining (MRDM) were often discretized, after considering the schema of the relational data base. Study the effects of taking the one-to-many association issue into consideration in the process of discretizing continuous numbers. Approach: Different alternatives for dealing with continuous attributes in MRDM were considered in this study, namely equal-width (EWD), Equal-Height (EH), equal-weight (EWG) and Entropy-Based (EB). The discretization procedures considered in this study included algorithms that were not depended on the multi-relational structure of the data and also tha t are sensitive to this structure. A new method of discretization, called the entropy instance-based ( EIB) discretization method was implemented and evaluated with respect to C4.5 on the two well-known multi-relational databases that include the Mutagenesis dataset and the Hepatitis dataset for D iscovery Challenge PKDD 2005. Results: When the number of bins, b, is big (b = 8), the entropy- instance-based discretization method produced bette r data summarization results compared to the other di scretization methods, in the mutagenesis dataset. I n contrast, for the hepatitis dataset, the entropy-in stance-based discretization method produced better data summarization results for all values of b, com pared to the other discretization methods. In the Hepatitis dataset, all discretization methods produ ced higher average performance accuracy (%) for partitional clustering technique, compared to the h ierarchical technique. Conclusion: These results demonstrated that entropy-based discretization can be improved by taking into consideration the multiple-instance problem. It was also found that t he partitional clustering technique produced better performance accuracy compared to the one produced by hierarchical clustering technique.

Data mining method for listed companies’ financial distress prediction

Data mining technique is capable of mining valuable knowledge from large and changeable database. This paper puts forward a data mining method combining attribute-oriented induction, information gain, and decision tree, which is suitable for preprocessing financial data and constructing decision tree model for financial distress prediction. On the base of financial ratios attributes and one class attribute, adopting entropy-based discretization method, a data mining model for listed companies’ financial distress prediction is designed. The empirical experiment with 35 financial ratios and 135 pairs of listed companies as initial samples got satisfying result, which testifies the feasibility and validity of the proposed data mining method for listed companies’ financial distress prediction.

A discretization algorithm based on a heterogeneity criterion

Discretization, as a preprocessing step for data mining, is a process of converting the continuous attributes of a data set into discrete ones so that they can be treated as the nominal features by machine learning algorithms. Those various discretization methods, that use entropy-based criteria, form a large class of algorithm. However, as a measure of class homogeneity, entropy cannot always accurately reflect the degree of class homogeneity of an interval. Therefore, in this paper, we propose a new measure of class heterogeneity of intervals from the viewpoint of class probability itself. Based on the definition of heterogeneity, we present a new criterion to evaluate a discretization scheme and analyze its property theoretically. Also, a heuristic method is proposed to find the approximate optimal discretization scheme. Finally, our method is compared, in terms of predictive error rate and tree size, with Ent-MDLC, a representative entropy-based discretization method well-known for its good performance. Our method is shown to produce better results than those of Ent-MDLC, although the improvement is not significant. It can be a good alternative to entropy-based discretization methods.