Classification Rule Discovery Research Articles

Discovery of Classification Rules Using Distributed Genetic Algorithm

This paper presents a distributed genetic algorithm for the discovery of classification rules. Population is contained in the form of interconnected demes. The local selection and reproduction mechanism is used to evolve the species within demes, and diversity is enhanced by migrating rules among some of the selected demes. Subsumption operator has been finally applied to reduce the complexity of the rule set discovered. The effectiveness of the proposed distributed genetic algorithm for discovering classification rules is evaluated by comparing the results with traditional crowding GA on 10 datasets from the UCI and KEEL repository. The results confirm that the distributed GA discover classification rules with significantly higher predictive accuracy. The influence of migration operator is also analysed with respect to migration rate.

Feature Selection Based Hybrid Anomaly Intrusion Detection System Using K Means and RBF Kernel Function

In Information Security, intrusion detection is the act of detecting actions that attempt to compromise the security goals. One of the primary challenges to intrusion detection is the problem of misjudgment, misdetection and lack of real time response to the attack. Various data mining techniques as clustering, classification and association rule discovery are being used for intrusion detection. The proposed hybrid technique combines data mining approaches like K Means clustering algorithm and RBF kernel function of Support Vector Machine as a classification module. The main purpose of proposed technique is to decrease the number of attributes associated with each data point. So, the proposed technique can perform better in terms of Detection Rate and Accuracy when applied to KDDCUP’99 Data Set.

Associative Classification Approaches: Review and Comparison

Associative classification (AC) is a promising data mining approach that integrates classification and association rule discovery to build classification models (classifiers). In the last decade, several AC algorithms have been proposed such as Classification based Association (CBA), Classification based on Predicted Association Rule (CPAR), Multi-class Classification using Association Rule (MCAR), Live and Let Live (L3) and others. These algorithms use different procedures for rule learning, rule sorting, rule pruning, classifier building and class allocation for test cases. This paper sheds the light and critically compares common AC algorithms with reference to the abovementioned procedures. Moreover, data representation formats in AC mining are discussed along with potential new research directions.

Classification Rule Discovery With Ant Colony Optimization

This paper proposes an algorithm for data mining called Ant-Miner (ant-colony-based data miner). The goal of Ant-Miner is to extract classification rules from data. The algorithm is inspired by both researches on the behavior of real ant colonies and some data mining concepts as well as principles. Recently research shows that ant colony optimization algorithm have been applied successfully to combinatorial optimization problems. In this paper we present an improvement to Ant-Miner. We compare the performance of new algorithm with before algorithm in two public domain data sets. Keyword : Ant Colony, Classification Rules, Data mining, Extract knowledge, Database.

Attribute Reduction based Hybrid Anomaly Intrusion Detection using K-Means and SVM Classifier

In Information Security, intrusion detection is the act of detecting actions that attempt to compromise the confidentiality, integrity or availability of a resource. One of the primary challenges to intrusion detection is the problem of misjudgment, misdetection and lack of real time response to the attack. Various data mining techniques such as clustering, classification and association rule discovery are being used for intrusion detection. The proposed hybrid technique combines data mining approaches like K Means clustering algorithm and Support Vector Machine classification module. The main purpose of proposed technique is to decrease the number of attributes associated with each data point. So that, the proposed technique can perform better in terms of Detection Rate and Accuracy when applied to KDD’99 Data Set.

Correlation as a Heuristic for Accurate and Comprehensible Ant Colony Optimization Based Classifiers

The primary objective of this research is to propose and investigate a novel ant colony optimization-based classification rule discovery algorithm and its variants. The main feature of this algorithm is a new heuristic function based on the correlation between attributes of a dataset. Several aspects and parameters of the proposed algorithm are investigated by experimentation on a number of benchmark datasets. We study the performance of our proposed approach and compare it with several state-of-the art commonly used classification algorithms. Experimental results indicate that the proposed approach builds more accurate models than the compared algorithms. The high accuracy supplemented by the comprehensibility of the discovered rule sets is the main advantage of this method.

Revisiting evolutionary algorithms in feature selection and nonfuzzy/fuzzy rule based classification

AbstractThis paper discusses the relevance and possible applications of evolutionary algorithms, particularly genetic algorithms, in the domain of knowledge discovery in databases. Knowledge discovery in databases is a process of discovering knowledge along with its validity, novelty, and potentiality. Various genetic‐based feature selection algorithms with their pros and cons are discussed in this article. Rule (a kind of high‐level representation of knowledge) discovery from databases, posed as single and multiobjective problems is a difficult optimization problem. Here, we present a review of some of the genetic‐based classification rule discovery methods based on fidelity criterion. The intractable nature of fuzzy rule mining using single and multiobjective genetic algorithms reported in the literatures is reviewed. An extensive list of relevant and useful references are given for further research. © 2013 Wiley Periodicals, Inc.This article is categorized under: Fundamental Concepts of Data and Knowledge > Key Design Issues in Data Mining Technologies > Computational Intelligence

Optimization of Decision Rules in Fuzzy Classification

are various advances in data collection that can intelligently and automatically analyze and mine knowledge from large amounts of data. World Wide Web as a global information system has flooded us with a tremendous amount of data and information Discovery of knowledge and decision-making directly from such huge volumes of data contents is a real challenge. The Knowledge Discovery in Databases (KDD) is the process of extracting the knowledge from huge data collection. Data mining is a step of KDD in which patterns or models are extracted from data by using some automated techniques. Discovering knowledge in the form of classification rules is one of the most important tasks of data mining. Discovery of comprehensible, concise and effective rules helps us to make right decisions. Therefore, several Machine Learning techniques are applied for discovery of classification rules. Recently there have been several applications of genetic algorithms for effective rules with high predictive accuracy.

Utilizing multiple pheromones in an ant-based algorithm for continuous-attribute classification rule discovery

The cAnt-Miner algorithm is an Ant Colony Optimization (ACO) based technique for classification rule discovery in problem domains which include continuous attributes. In this paper, we propose several extensions to cAnt-Miner. The main extension is based on the use of multiple pheromone types, one for each class value to be predicted. In the proposed μcAnt-Miner algorithm, an ant first selects a class value to be the consequent of a rule and the terms in the antecedent are selected based on the pheromone levels of the selected class value; pheromone update occurs on the corresponding pheromone type of the class value. The pre-selection of a class value also allows the use of more precise measures for the heuristic function and the dynamic discretization of continuous attributes, and further allows for the use of a rule quality measure that directly takes into account the confidence of the rule. Experimental results on 20 benchmark datasets show that our proposed extension improves classification accuracy to a statistically significant extent compared to cAnt-Miner, and has classification accuracy similar to the well-known Ripper and PART rule induction algorithms.

A novel chemistry based metaheuristic optimization method for mining of classification rules

When investigated carefully, chemical reactions possess efficient objects, states, process, and events that can be designed as a computational method en bloc. In this study, a novel computational method, which is robust and have less parameters than that of used in the literature, is intended to be developed inspiring from types and occurring of chemical reactions. The proposed method is named as artificial chemical reaction optimization algorithm, ACROA. In this study, one of the first applications of this method has been performed in classification rule discovery field of data mining and efficiency has been demonstrated.

A Hybrid PSO with Dynamic Inertia Weight and GA Approach for Discovering Classification Rule in Data Mining

Mining is the efficient knowledge discovery form database. It is also form of knowledge discovery essential for solving problem in specific domain like health care, business and other field. The proposed system is based on population based on heuristic search technique, which can used to solve combinatorial optimization problem. Our research focus on studying the hybrid algorithm that result in performance and enhancement in classification rule discovery task. In standard Particle Swarm Optimization (PSO) the non oscillatory route can quickly cause a particle to stagnate and also it may prematurely converge on suboptimal solution that is not even guaranteed to local optimal solution. In this paper we have present novel hybrid algorithm, PSO with Dynamic Inertia Weight and Genetic Algorithm (GA) approach for classification rule. The selection of inertia weight was very important to ensure the convergent behavior of particle In this hybrid algorithm approach incorporates a dynamic inertia weight in order to help the algorithm to find global and overcome the problem convergence to local optima, essentially GA can perform a global search over the entire search space with faster convergence speed. Thus the hybrid algorithm is easily implemented because of use of simple classifier it has, its computational complexity is low, are the special characteristics for the use of this hybrid algorithm. KeywordsAlgorithm; Particle Swarm Optimization; Classification;

FUZZY GRAVITATIONAL SEARCH ALGORITHM AN APPROACH FOR DATA MINING

The concept of intelligently controlling the search process of grav- itational search algorithm (GSA) is introduced to develop a novel data mining technique. The proposed method is called fuzzy GSA miner (FGSA-miner). At first a fuzzy controller is designed for adaptively controlling the gravitational coefficient and the number of effective objects, as two important parameters which play major roles on search process of GSA. Then the improved GSA (namely Fuzzy-GSA) is employed to construct a novel data mining algorithm for classification rule discovery from reference data sets. Extensive experimen- tal results on different benchmarks and a practical pattern recognition problem with nonlinear, overlapping class boundaries and different feature space di- mensions are provided to show the powerfulness of the proposed method. The comparative results illustrate that performance of the proposed FGSA-miner considerably outperforms the standard GSA. Also it is shown that the perfor- mance of the FGSA-miner is comparable to, sometimes better than those of the CN2 (a traditional data mining method) and similar approach which have been designed based on other swarm intelligence algorithms (ant colony opti- mization and particle swarm optimization) and evolutionary algorithm (genetic algorithm).

Multiple pheromone types and other extensions to the Ant-Miner classification rule discovery algorithm

Ant-Miner is an ant-based algorithm for the discovery of classification rules. This paper proposes five extensions to Ant-Miner: (1) we utilize multiple types of pheromone, one for each permitted rule class, i.e. an ant first selects the rule class and then deposits the corresponding type of pheromone; (2) we use a quality contrast intensifier to magnify the reward of high-quality rules and to penalize low-quality rules in terms of pheromone update; (3) we allow the use of a logical negation operator in the antecedents of constructed rules; (4) we incorporate stubborn ants, an ACO variation in which an ant is allowed to take into consideration its own personal past history; (5) we use an ant colony behavior in which each ant is allowed to have its own values of the α and β parameters (in a sense, to have its own personality). Empirical results on 23 datasets show improvements in the algorithm’s performance in terms of predictive accuracy and simplicity of the generated rule set.

Analysis of powered two-wheeler crashes in Italy by classification trees and rules discovery

Aim of the study was the analysis of powered two-wheeler (PTW) crashes in Italy in order to detect interdependence as well as dissimilarities among crash characteristics and provide insights for the development of safety improvement strategies focused on PTWs. At this aim, data mining techniques were used to analyze the data relative to the 254,575 crashes involving PTWs occurred in Italy in the period 2006–2008.Classification trees analysis and rules discovery were performed. Tree-based methods are non-linear and non-parametric data mining tools for supervised classification and regression problems. They do not require a priori probabilistic knowledge about the phenomena under studying and consider conditional interactions among input data. Rules discovery is the identification of sets of items (i.e., crash patterns) that occur together in a given event (i.e., a crash in our study) more often than they would if they were independent of each other. Thus, the method can detect interdependence among crash characteristics. Due to the large number of patterns considered, both methods suffer from an extreme risk of finding patterns that appear due to chance alone. To overcome this problem, in our study we randomly split the sample data in two data sets and used well-established statistical practices to evaluate the statistical significance of the results.Both the classification trees and the rules discovery were effective in providing meaningful insights about PTW crash characteristics and their interdependencies. Even though in several cases different crash characteristics were highlighted, the results of the two the analysis methods were never contradictory. Furthermore, most of the findings of this study were consistent with the results of previous studies which used different analytical techniques, such as probabilistic models of crash injury severity. Basing on the analysis results, engineering countermeasures and policy initiatives to reduce PTW injuries and fatalities were singled out. The simultaneous use of classification trees and association discovery must not, however, be seen as an attempt to supplant other techniques, but as a complementary method which can be integrated into other safety analyses.

Modeling and implementation of classification rule discovery by ant colony optimisation for spatial land-use suitability assessment

This paper presents an integrated modeling method for multi-criteria land-use suitability assessment (LSA) using classification rule discovery (CRD) by ant colony optimisation (ACO) in ArcGIS. This new attempt applies artificial intelligent algorithms to intelligentise LSA by discovering suitability classification rules. The methodology is implemented as a tool called ACO–LSA. The tool can generate rules which are straightforward and comprehensible for users with high classification accuracy and simple rule list in solving CRD problems. A case study in the Macintyre Brook Catchment of southern Queensland in Australia is proposed to demonstrate the feasibility of this new modeling technique. The results have addressed the major advantages of this novel approach.

A Fast Boosting Based Incremental Genetic Algorithm for Mining Classification Rules in Large Datasets

Genetic algorithm is a search technique purely based on natural evolution process. It is widely used by the data mining community for classification rule discovery in complex domains. During the learning process it makes several passes over the data set for determining the accuracy of the potential rules. Due to this characteristic it becomes an extremely I/O intensive slow process. It is particularly difficult to apply GA when the training data set becomes too large and not fully available. An incremental Genetic algorithm based on boosting phenomenon is proposed in this paper which constructs a weak ensemble of classifiers in a fast incremental manner and thus tries to reduce the learning cost considerably.

A correlation-based ant miner for classification rule discovery

In recent years, a few sequential covering algorithms for classification rule discovery based on the ant colony optimization meta-heuristic (ACO) have been proposed. This paper proposes a new ACO-based classification algorithm called AntMiner-C. Its main feature is a heuristic function based on the correlation among the attributes. Other highlights include the manner in which class labels are assigned to the rules prior to their discovery, a strategy for dynamically stopping the addition of terms in a rule’s antecedent part, and a strategy for pruning redundant rules from the rule set. We study the performance of our proposed approach for twelve commonly used data sets and compare it with the original AntMiner algorithm, decision tree builder C4.5, Ripper, logistic regression technique, and a SVM. Experimental results show that the accuracy rate obtained by AntMiner-C is better than that of the compared algorithms. However, the average number of rules and average terms per rule are higher.

Classification rule discovery with DE/QDE algorithm

The quantum-inspired differential evolution algorithm (QDE) is a new optimization algorithm in the binary-valued space. The paper proposes the DE/QDE algorithm for the discovery of classification rules. DE/QDE combines the characteristics of the conventional DE algorithm and the QDE algorithm. Based on some strategies of DE and QDE, DE/QDE can directly cope with the continuous, nominal attributes without discretizing the continuous attributes in the preprocessing step. DE/QDE also has specific weight mutation for managing the weight value of the individual encoding. Then DE/QDE is compared with Ant-Miner and CN2 on six problems from the UCI repository datasets. The results indicate that DE/QDE is competitive with Ant-Miner and CN2 in term of the predictive accuracy.

Classification rule discovery for the aviation incidents resulted in fatality

Data mining methods have been successfully applied to different fields. Aviation industry is one of them. There is a large amount of knowledge and data accumulation in aviation industry. These data could be stored in the form of pilot reports, maintenance reports, incident reports or delay reports. This paper explains the data mining application on the incident reports of the Federal Aviation Administration (FAA) Accident/Incident Data System database, contains incident data records for all categories of civil aviation between the years of 2000 and 2006. In this study, we applied data mining methods on the incident reports. Moreover rough sets concept is used to reduce the attributes of data set. The purpose of this application is to find out the effective attributes in order to reduce the number of the fatality in the incidents. The categorization tools and decision trees are used to find the relations and rules about the incidents resulted in fatality. For this purpose data-mining analysis is conducted. As a result some rules about the fatality are obtained and also the parameters that affect the fatality in the incident have determined. The rules found are tested in terms of their accuracy and reliability, and these results are seen to be meaningful.

A Rough Set Approach for the Discovery of Classification Rules in Interval-Valued Information Systems

A novel rough set approach is proposed in this paper to discover classification rules through a process of knowledge induction which selects decision rules with a minimal set of features for classification of real-valued data. A rough set knowledge discovery framework is formulated for the analysis of interval-valued information systems converted from real-valued raw decision tables. The minimal feature selection method for information systems with interval-valued features obtains all classification rules hidden in a system through a knowledge induction process. Numerical examples are employed to substantiate the conceptual arguments.