Extract Classification Rules Research Articles

Classification Rule Extraction Based on Fuzzy Area Distribution and Classification Reasoning Algorithm

A definition of rule relative compatibility is proposed based on membership degree distribution on the fuzzy area corresponding to every rule.Compared with the class compatibility,rule relative compatibility is able to show more information about the samples distribution difference among every fuzzy area.The Fuzzy Area Distribution Matrix is designed,by which the rule relative compatibility and class compatibility are calculated respectively.Moreover,the algorithm of classification rule extraction is put forward by rule relative compatibility.Different from the class compatibility approach,the one based on rule relative compatibility contains the membership degree distribution comparison of every rule.Furthermore,this compatibility is weighted by the relative amount of every class samples so that it can take the consideration of the global density dominance as well as the local quantity dominance for study space.In addition,the classification reasoning based on fuzzy rule is implemented by the defuzzifier algorithm.The procedure is better than the previous algorithm due to its interpretability and simplicity.In the end,Iris data and Wine data are used to validate the proposed algorithm of fuzzy classification rule extraction.The testing results prove that whether the sample is distributed homogeneously or not,the rule extraction approach based on rule relative compatibility attains higher classification rate.

A soft computing-based approach for integrated training and rule extraction from artificial neural networks: DIFACONN-miner

Artificial neural network (ANN) is one of the most widely used techniques in classification data mining. Although ANNs can achieve very high classification accuracies, their explanation capability is very limited. Therefore one of the main challenges in using ANNs in data mining applications is to extract explicit knowledge from them. Based on this motivation, a novel approach is proposed in this paper for generating classification rules from feed forward type ANNs. Although there are several approaches in the literature for classification rule extraction from ANNs, the present approach is fundamentally different from them. In the previous studies, ANN training and rule extraction is generally performed independently in a sequential (hierarchical) manner. However, in the present study, training and rule extraction phases are integrated within a multiple objective evaluation framework for generating accurate classification rules directly. The proposed approach makes use of differential evolution algorithm for training and touring ant colony optimization algorithm for rule extracting. The proposed algorithm is named as DIFACONN-miner. Experimental study on the benchmark data sets and comparisons with some other classical and state-of-the art rule extraction algorithms has shown that the proposed approach has a big potential to discover more accurate and concise classification rules.

TACO-miner: An ant colony based algorithm for rule extraction from trained neural networks

Extracting classification rules from data is an important task of data mining and gaining considerable more attention in recent years. In this paper, a new meta-heuristic algorithm which is called as TACO-miner is proposed for rule extraction from artificial neural networks (ANN). The proposed rule extraction algorithm actually works on the trained ANNs in order to discover the hidden knowledge which is available in the form of connection weights within ANN structure. The proposed algorithm is mainly based on a meta-heuristic which is known as touring ant colony optimization (TACO) and consists of two-step hierarchical structure. The proposed algorithm is experimentally evaluated on six binary and n-ary classification benchmark data sets. Results of the comparative study show that TACO-miner is able to discover accurate and concise classification rules.

Generating prediction rules for liquefaction through data mining

Prediction of liquefaction is an important subject in geotechnical engineering. Prediction of liquefaction is also a complex problem as it depends on many different physical factors, and the relations between these factors are highly non-linear and complex. Several approaches have been proposed in the literature for modeling and prediction of liquefaction. Most of these approaches are based on classical statistical approaches and neural networks. In this paper a new approach which is based on classification data mining is proposed first time in the literature for liquefaction prediction. The proposed approach is based on extracting accurate classification rules from neural networks via ant colony optimization. The extracted classification rules are in the form of IF–THEN rules which can be easily understood by human. The proposed algorithm is also compared with several other data mining algorithms. It is shown that the proposed algorithm is very effective and accurate in prediction of liquefaction.

Construction litigation prediction system using ant colony optimization

The frequency of construction litigation has increased over the years, making litigation a costly and time‐consuming activity. It is in the interests of all parties to a construction contract to avoid litigation. A tool (Ant Miner) is proposed to predict the outcome of construction litigation, hence encouraging the parties to settle out of court. Ant Miner, a rule‐based classification system extracts classification rules by using ant colony optimization. It is used on 151 Illinois circuit court cases filed in the period 1987–2005. The prediction model is composed of data consolidation, attribute selection, classification and assessment. The results provide evidence that Ant Miner performs better than models used in earlier studies and that the rule sets discovered by this tool are highly interpretable, but that this tool suffers a great deal from noisy data. If the parties involved in a dispute have access to the proposed system that predicts the decision of the courts with higher accuracy and reliability than before, then they are expected to avoid litigation and settle out of court in order to save considerable time, money and aggravation.

Recursive Neural Network Rule Extraction for Data With Mixed Attributes

In this paper, we present a recursive algorithm for extracting classification rules from feedforward neural networks (NNs) that have been trained on data sets having both discrete and continuous attributes. The novelty of this algorithm lies in the conditions of the extracted rules: the rule conditions involving discrete attributes are disjoint from those involving continuous attributes. The algorithm starts by first generating rules with discrete attributes only to explain the classification process of the NN. If the accuracy of a rule with only discrete attributes is not satisfactory, the algorithm refines this rule by recursively generating more rules with discrete attributes not already present in the rule condition, or by generating a hyperplane involving only the continuous attributes. We show that for three real-life credit scoring data sets, the algorithm generates rules that are not only more accurate but also more comprehensible than those generated by other NN rule extraction methods.

Discovering Unordered Rule Sets for Mixed Variables Using an Ant-Miner Algorithm

This work proposes a data mining algorithm called Unordered Rule Sets using a continuous Ant-Miner algorithm. The goal of this work is to extract classification rules from data. Swarm intelligence (SI) is a technique whereby rules may be discovered through the study of collective behavior in decentralized, self-organized systems, such as ants. The Ant-Miner algorithm, first proposed by Parpinelli and his colleagues (2002), applies an ant colony optimization (ACO) heuristic to the classification task of data mining to discover an ordered list of classification rules. Ant-Miner is a rule-induction algorithm that uses SI techniques to form rules. Ant-Miner uses a discretization process to deal with continuous attributes in the data. Discretization transforms numeric attributes into nominal attributes. Discretization may suffer from a loss of information, as the real relationship underlying individual values of a numeric attribute is unknown. The objective of this work is to apply ACO heuristic techniques to discover unordered rule sets for mixed variables in a data set. The proposed algorithm handles both nominal and continuous attributes using multimodal functions. It has the advantage of discovering more modular rules, i.e., rules that can be interpreted independently from other rules - unlike the rules in an ordered list, where the interpretation of a rule requires knowledge of the previous rules in the list. The results provide evidence that the accuracy of the Unordered Rule Set Continuous Ant-Miner algorithm is competitive with other Ant-Miner versions and generates simpler rule sets.

Empirical studies to assess the understandability of data warehouse schemas using structural metrics

Data warehouses are powerful tools for making better and faster decisions in organizations where information is an asset of primary importance. Due to the complexity of data warehouses, metrics and procedures are required to continuously assure their quality. This article describes an empirical study and a replication aimed at investigating the use of structural metrics as indicators of the understandability, and by extension, the cognitive complexity of data warehouse schemas. More specifically, a four-step analysis is conducted: (1) check if individually and collectively, the considered metrics can be correlated with schema understandability using classical statistical techniques, (2) evaluate whether understandability can be predicted by case similarity using the case-based reasoning technique, (3) determine, for each level of understandability, the subsets of metrics that are important by means of a classification technique, and assess, by means of a probabilistic technique, the degree of participation of each metric in the understandability prediction. The results obtained show that although a linear model is a good approximation of the relation between structure and understandability, the associated coefficients are not significant enough. Additionally, classification analyses reveal respectively that prediction can be achieved by considering structure similarity, that extracted classification rules can be used to estimate the magnitude of understandability, and that some metrics such as the number of fact tables have more impact than others.

Extraction of classification rules characterized by ellipsoidal regions using soft-computing techniques

This article presents a soft-computing-based data mining technique that addresses methodology aspects on extracting classification rules characterized by ellipsoidal regions in feature space. Self-organizing mapping and statistical techniques are employed to initialize the rules. A regularization model embedding some information on recognition rate and generalization ability is presented for refining the initial rules. Rule optimization is implemented for each individual rule using an evolutionary strategy. To generate rules for patterns with low probability of occurrence but considerable conceptual importance, a multilayer structure of rule generation and use is proposed. Simulation results are carried out by three benchmark data sets, and compared with other data mining tools and classifiers, such as decision trees, BRAINNE (Building Representations of Artificial Intelligence (AI) using Neural Networks), support vector machine, and neural networks. Our technique demonstrates its power and potential for real-world applications.

Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability

The generation of predictive models is a frequent task in data mining with the objective of generating highly precise and interpretable models. The data reduction is an interesting preprocessing approach that can allow us to obtain predictive models with these characteristics in large size data sets. In this paper, we analyze the rule classification model based on decision trees using a training selected set via evolutionary stratified instance selection. This method faces the scaling problem that appears in the evaluation of large size data sets, and the trade off interpretability-precision of the generated models.

Entropy-based fuzzy rough classification approach for extracting classification rules

Abstract Recently, the combination of Fuzzy Set Theory and Rough Set Theory has become a popular data mining technique for classification problems because of their strength of handling vague and imprecise data. From the previous literature, Rough Set Theory can only operate effectively with datasets containing discrete values. As most datasets contain real-valued features, it is necessary to perform a discretization step beforehand, which is typically implemented by standard fuzzification techniques. In this paper, a new fuzzification technique called Modified Minimization Entropy Principle Algorithm (MMEPA) is proposed to construct membership functions of fuzzy sets of linguistic variables. Using the dataset fuzzified by this technique to perform the rule extraction algorithm Variable Precision Rough Set Model (VP-model), the extracted classification rules by this model can obtain a higher classification accuracy rate than that of some existing methods.

Ensemble classifier system based on ant colony algorithm and its application in chemical pattern classification

A novel ant colony algorithm, mass recruitment and group recruitment based continuous ant colony optimization (MG-CACO), is proposed to solve continuous optimization problems. MG-CACO, which can capture the interdependencies between attributes and does not need discretization as a preprocessing step for optimization, was applied to extract classification rules from samples. To improve the predictive performance of the classifier, the ensemble strategy was adopted and the MG-CACO based ensemble classifier system called MG-CACO-ECS was built. Several datasets, obtained from UCI (University of California, Irvine) machine learning repository, were employed to illustrate the validity of MG-CACO-ECS. The results indicated that MG-CACO-ECS has satisfactory prediction accuracy. Furthermore, the problem of the producing area discrimination of olive oil was studied, and the obtained results demonstrated that MG-CACO-ECS has better prediction accuracy than the reported results.

Essential classification rule sets

Given a class model built from a dataset including labeled data, classification assigns a new data object to the appropriate class. In associative classification the class model (i.e., the classifier) is a set of association rules. Associative classification is a promising technique for the generation of highly accurate classifiers. In this article, we present a compact form which encodes without information loss the classification knowledge available in a classification rule set. This form includes the rules that are essential for classification purposes, and thus it can replace the complete rule set. The proposed form is particularly effective in dense datasets, where traditional extraction techniques may generate huge rule sets. The reduction in size of the rule set allows decreasing the complexity of both the rule generation step and the rule pruning step. Hence, classification rule extraction can be performed also with low support, in order to extract more, possibly useful, rules.

Knowledge Discovery From Multispectral Satellite Images

A new approach to extract knowledge from multispectral images is suggested. We describe a method to extract and optimize classification rules using fuzzy neural networks (FNNs). The FNNs consist of two stages. The first stage represents a fuzzifier block, and the second stage represents the inference engine. After training, classification rules are extracted by backtracking along the weighted paths through the FNN. The extracted rules are then optimized by use of a fuzzy associate memory bank. We use the algorithm to extract classification rules from a multispectral image obtained with a Landsat Thematic Mapper sensor. The scene represents the Mississippi River bottomland area. In order to verify the rule extraction method, measures such as the overall accuracy, producer's accuracy, user's accuracy, kappa coefficient, and fidelity are used.

A feature selection bayesian approach for extracting classification rules with a clustering genetic algorithm

This work describes a method that combines a Bayesian feature selection approach with a clustering genetic algorithm to get classification rules in data-mining applications. A Bayesian network is generated from a data set and the Markov blanket of the class variable is applied to the feature subset selection task. The general rule extraction method is simple and consists of employing the clustering process in the examples of each class separately. In this way, clusters of similar examples are found for each class. These clusters can be viewed as subclasses and can, consequently, be modeled into logical rules. In this context, the problem of finding the optimal number of classification rules can be viewed as the problem of finding the best number of clusters. The Clustering Genetic Algorithm can find the best clustering in a data set, according to the Average Silhouette Width criterion, and it was applied to extract classification rules. The proposed methodology is illustrated by means of simulations in three data sets that are benchmarks for data-mining methods--Wisconsin Breast Cancer, Mushroom, and Congressional Voting Records. The rules extracted with all the attributes are compared to those extracted with the features belonging to the Markov blanket and the obtained results show that the proposed method is very promising.

Evolutionary computing for knowledge discovery in medical diagnosis

One of the major challenges in medical domain is the extraction of comprehensible knowledge from medical diagnosis data. In this paper, a two-phase hybrid evolutionary classification technique is proposed to extract classification rules that can be used in clinical practice for better understanding and prevention of unwanted medical events. In the first phase, a hybrid evolutionary algorithm (EA) is utilized to confine the search space by evolving a pool of good candidate rules, e.g. genetic programming (GP) is applied to evolve nominal attributes for free structured rules and genetic algorithm (GA) is used to optimize the numeric attributes for concise classification rules without the need of discretization. These candidate rules are then used in the second phase to optimize the order and number of rules in the evolution for forming accurate and comprehensible rule sets. The proposed evolutionary classifier (EvoC) is validated upon hepatitis and breast cancer datasets obtained from the UCI machine-learning repository. Simulation results show that the evolutionary classifier produces comprehensible rules and good classification accuracy for the medical datasets. Results obtained from t-tests further justify its robustness and invariance to random partition of datasets.

データマイニングを目的とした遺伝的プログラミングの拡張による分類学習システムとルール生成システム

本研究では、遺伝的プログラミング(以下、GP)を用いたデータマイニングについて論じたものであり、GPと分類学習手法の組み合せによるデータマイニング手法を提案し、これについての有効性の検証を行った。1章の序論、2章、提案する手法の核となるGPについて説明に続き、3章では、GPと決定本構築法を組み合わせる手法を提案し、4章では、GPと相関ルール抽出法を組み合わせる手法を提案し、5章では、GPと論理関数を組み合わせた手法を提案した。6章では、GPに変更を加えることにより、GPが大規模データマイニングにおいても十分に利用可能であることを明らかにした。改善手法として、オブジェクト指向の組み合わせ学習、無効ノード削除と連続値属性適応処理の2つの強化学習を組み合わせる方法、交叉個体を複数指標により選ぶ方法の3つを提案した。7章は、本論文における提案手法を総括し、さらなる改善の可能性について検討した。

Data mining with an ant colony optimization algorithm

The paper proposes an algorithm for data mining called Ant-Miner (ant-colony-based data miner). The goal of Ant-Miner is to extract classification rules from data. The algorithm is inspired by both research on the behavior of real ant colonies and some data mining concepts as well as principles. We compare the performance of Ant-Miner with CN2, a well-known data mining algorithm for classification, in six public domain data sets. The results provide evidence that: 1) Ant-Miner is competitive with CN2 with respect to predictive accuracy, and 2) the rule lists discovered by Ant-Miner are considerably simpler (smaller) than those discovered by CN2.

Generating concise and accurate classification rules for breast cancer diagnosis

In our previous work, we have presented an algorithm that extracts classification rules from trained neural networks and discussed its application to breast cancer diagnosis. In this paper, we describe how the accuracy of the networks and the accuracy of the rules extracted from them can be improved by a simple pre-processing of the data. Data pre-processing involves selecting the relevant input attributes and removing those samples with missing attribute values. The rules generated by our neural network rule extraction algorithm are more concise and accurate than those generated by other rule generating methods reported in the literature.

修正忘却付き構造学習と並列多層ネットワークを用いた規則発見

修正忘却付き構造学習と並列多層ネットワークを用いた規則発見