Discovering Classification Rules Research Articles

Genetic Biomarkers of Sorafenib Response in Patients with Hepatocellular Carcinoma.

The identification of biomarkers for predicting inter-individual sorafenib response variability could allow hepatocellular carcinoma (HCC) patient stratification. SNPs in angiogenesis- and drug absorption, distribution, metabolism, and excretion (ADME)-related genes were evaluated to identify new potential predictive biomarkers of sorafenib response in HCC patients. Five known SNPs in angiogenesis-related genes, including VEGF-A, VEGF-C, HIF-1a, ANGPT2, and NOS3, were investigated in 34 HCC patients (9 sorafenib responders and 25 non-responders). A subgroup of 23 patients was genotyped for SNPs in ADME genes. A machine learning classifier method was used to discover classification rules for our dataset. We found that only the VEGF-A (rs2010963) C allele and CC genotype were significantly associated with sorafenib response. ADME-related gene analysis identified 10 polymorphic variants in ADH1A (rs6811453), ADH6 (rs10008281), SULT1A2/CCDC101 (rs11401), CYP26A1 (rs7905939), DPYD (rs2297595 and rs1801265), FMO2 (rs2020863), and SLC22A14 (rs149738, rs171248, and rs183574) significantly associated with sorafenib response. We have identified a genetic signature of predictive response that could permit non-responder/responder patient stratification. Angiogenesis- and ADME-related genes correlation was confirmed by cumulative genetic risk score and network and pathway enrichment analysis. Our findings provide a proof of concept that needs further validation in follow-up studies for HCC patient stratification for sorafenib prescription.

Privacy preserving rule-based classifier using modified artificial bee colony algorithm

Privacy preserving data mining is a hot research field of data mining. The aim of privacy preserving data mining is to prevent the leakage of the sensitive information of individuals while performing data mining techniques. Classification task is one of the most studied fields in data mining hence in privacy preserving data mining as well. On the other hand, differential privacy is a powerful privacy guarantee that determines privacy leakage ratio by using ∊ parameter and enables researchers to mine data which includes sensitive information. Implementations of some well-known classification algorithms such as k-NN, Naïve Bayes, ID3, etc. with differential privacy have been developed. Although the success of the rule-based classifiers using meta-heuristics such as Ant-Miner, BeeMiner etc. in data mining has been demonstrated, any implementation of these classification algorithms with differential privacy has not been proposed in the literature until now to our best knowledge. Artificial bee colony (ABC) is a nature inspired algorithm which imitates foraging behavior of bees, and some approaches using ABC to discover classification rules have been proposed recently and the success of ABC algorithm for the discovery of classification rules has been demonstrated. Motivated by this shortcoming in the literature, we propose to develop a rule-based classifier using ABC algorithm with input perturbation technique of differential privacy to perform privacy preserving classification. According to our experimental results, the proposed ABC-based classifier performs better than the well-known algorithms that are SVM, C4.5, Holte’s One Rule, PART, and RIPPER over non-private and differentially private versions of the datasets in terms of classification performance.

Ant Miner

Discovering classification rules from large data is an important task of data mining and is gaining considerable attention. This article presents a novel ant miner for classification rule mining. Our ant miner is inspired by research on the behavior of real ant colonies, simulated annealing, and some data mining concepts as well as principles. Here we present a Michigan style approach for single objective classification rule mining. The algorithm is tested on a few benchmark datasets drawn from UCI repository. Our experimental outcomes confirm that ant miner-HMC (Hybrid Michigan Style Classification) is significantly better than ant-miner-MC (Michigan Style Classification).

An Application of J48 Classification Algorithm in Predicting Students’ Academic Performance

This paper sets out to use J48 classification algorithm to predict students’ academic performance towards the end of the semester in the Data Structure course under the Computer Science Program. This algorithm aimed to help faculty in forecasting who among the students would likely to fail and who would make it until the end of the semester. In this way, the faculty could make remedial measures to help those struggling students pass the subject and advance to the next level, thus, increasing students’ success rate and retention in a Higher Education Institutions (HEI). This research employed a descriptive correlational design using Exploratory Data Analysis (EDA) for Data Mining in testing and verifying data to generate new information. Data mining is part of the Knowledge Discovery in Databases (KDD) process where it follows six steps: data selection, data pre-processing, data transformation, data mining, interpretation, and knowledge discovery. Step 1 includes gathering and selecting data for the study and for this purpose, a total of 103 students’ records were collected from the instructors for a period of two semesters, S.Y. 2014 -2015 & 2015 – 2016. Different evaluative criteria contained in the class records were utilized as attributes in predicting students’ academic performance. Steps 2 and 3 is pre-processing and transforming the data where it involves discarding those students who dropped/withdrawn from the semester, and converting the excel file into a comma separated values or .csv file, respectively. After these steps, step 4 or the application of J48 classification algorithm was utilized to discover classification rules. Step 5 refers to the tree visualization results where it identified the strongest predictor that most likely influence the students’ final average grade. Finally, step 7 shows the extracted information from the tree or the extracted rules that can be used by the administration, faculty and other stakeholders to improve the academic performance of the students. In particular, they might consider redesigning and restructuring teaching pedagogies to assist and focus more on struggling students.

Hybrid particle swarm optimization for rule discovery in the diagnosis of coronary artery disease

AbstractCoronary artery disease (CAD) is one of the major causes of mortality worldwide. Knowledge about risk factors that increase the probability of developing CAD can help to understand the disease better and assist in its treatment. Recently, modern computer‐aided approaches have been used for the prediction and diagnosis of diseases. Swarm intelligence algorithms like particle swarm optimization (PSO) have demonstrated great performance in solving different optimization problems. As rule discovery can be modelled as an optimization problem, it can be mapped to an optimization problem and solved by means of an evolutionary algorithm like PSO. An approach for discovering classification rules of CAD is proposed. The work is based on the real‐world CAD data set and aims at the detection of this disease by producing the accurate and effective rules. The proposed algorithm is a hybrid binary‐real PSO, which includes the combination of categorical and numerical encoding of a particle and a different approach for calculating the velocity of particles. The rules were developed from randomly generated particles, which take random values in the range of each attribute in the rule. Two different feature selection methods based on multi‐objective evolutionary search and PSO were applied on the data set, and the most relevant features were selected by the algorithms. The accuracy of two different rule sets were evaluated. The rule set with 11 features obtained more accurate results than the rule set with 13 features. Our results show that the proposed approach has the ability to produce effective rules with highest accuracy for the detection of CAD.

Exception discovery using ant colony optimisation

Ant colony optimisation (ACO) algorithms have been used to discover accurate, and comprehensible classification rules. Discovering exceptions using ACO is an underexplored area of research. Most of the classification algorithms focus on discovering rules with high generality. Since exceptions have low support, these often get ignored as noise. This paper proposes an ant colony optimisation (ACO)-based algorithm to discover classification rules in if-then-unless framework, where the unless part contains exceptions. We have conducted experiments on ten datasets from the UCI machine learning repository. The suggested algorithm is found to be competitive with the two well-known ACO-based classification algorithms (Ant-Miner and cAnt-MinerPB) with respect to predictive accuracy and comprehensibility. The algorithm has been able to capture a number of exceptions across several datasets. The classification rules discovered with exceptions are accurate, semantically comprehensible and interesting. These rules provide an opportunity to amend one's decision in exceptional circumstances.

A gene expression programming algorithm for discovering classification rules in the multi-objective space

Multi-objective evolutionary algorithms have been criticized when they are applied to classification rule mining, and, more specifically, in the optimization of more than two objectives due to their computational complexity. It is known that a multi-objective space is much richer to be explored than a single-objective space. In consequence, there are only few multi-objective algorithms for classification and their empirical assessed is quite limited. On the other hand, gene expression programming has emerged as an alternative to carry out the evolutionary process at genotypic level in a really efficient way. This paper introduces a new multi-objective algorithm for discovering classification rules, AR-NSGEP (Adaptive Reference point based Non-dominated Sorting with Gene Expression Programming). It is a multi-objective evolution of a previous single-objective algorithm. In AR-NSGEP, the multi-objective search was based on the well-known R-NSGA-II algorithm, replacing GA with GEP technology. Four objectives led the rules-discovery process, three of them (sensitivity, specificity and precision) were focused on promoting accuracy and the fourth (simpleness) on the interpretability of rules. AR-NSGEP was evaluated on several benchmark data sets and compared against six rule-based classifiers widely used. The AR-NSGEP, with four-objectives, achieved a significant improvement of the AUC metric with espect to most of the algorithms assessed, while the predictive accuracy and number of rules in the obtained models reached to acceptable results.

A Novel Fitness Computation Framework for Nature Inspired Classification Algorithms

Nature inspired algorithms have become popular for discovering classification rules due to their ability to effectively handle large and complex search spaces. However, nature inspired algorithms have to compute the fitness of individuals (candidate classification rules) over successive generations repeatedly. Each fitness computation requires scanning the training data. Since the database scan is computationally expensive operation, the execution time for nature inspired algorithms for discovering classification rules grows unreasonably faster for bulky datasets. This paper proposes a novel fitness computation framework for nature inspire algorithms by using a list structure. The indices of instances, covered by every possible attribute-value pair with respect to each class in the training data, are stored in the suggested list structure. The list is prepared only once in advance and stores all information to compute the fitness of any rule that may come up in the life time of the nature inspired algorithm. The existence of the pre-maintained list eliminates the need of scanning training data again and again for fitness computation. We have conducted experiments on 12 datasets from UCI machine learning repository. The results show that the suggested fitness computational framework brings significant speed gain without compromising predictive accuracy. Although, a Genetic Algorithm is used for classification rule discovery as the nature inspired algorithm in this paper, the fitness computation framework is general and can be used with any other nature inspired algorithm.

A fuzzy rule induction algorithm for discovering classification rules

© 2016 - IOS Press and the authors. All rights reserved. Recently, the topic of data mining has attracted considerable attention from both academia and industry. Data mining is the process of extracting useful knowledge from large amounts of data. Among the types of knowledge to be mined, classification knowledge is the most widely exploited in engineering applications. A variety of methods exist for inductive learning of crisp classification knowledge. This paper presents a new inductive learning algorithm called FuzzyRULES that extracts fuzzy classification rules from a database of examples. The use of fuzzy sets and fuzzy logic methods not only provides a powerful, flexible approach to handling vagueness and uncertainty, but also increases the expressive power and comprehensibility of the induced classification knowledge. An example involving the induction of process planning rules is used to illustrate the operation of FuzzyRULES. The algorithm has also been compared against conventional crisp and fuzzy rule induction algorithms on several benchmark data sets. The results obtained have shown that FuzzyRULES induces more compact and more accurate classification rule sets.

CoABCMiner: An Algorithm for Cooperative Rule Classification System Based on Artificial Bee Colony

In data mining, classification rule learning extracts the knowledge in the representation of IF_THEN rule which is comprehensive and readable. It is a challenging problem due to the complexity of data sets. Various meta-heuristic machine learning algorithms are proposed for rule learning. Cooperative rule learning is the discovery process of all classification rules with a single run concurrently. In this paper, a novel cooperative rule learning algorithm, called CoABCMiner, based on Artificial Bee Colony is introduced. The proposed algorithm handles the training data set and discovers the classification model containing the rule list. Token competition, new updating strategy used in onlooker and employed phases, and new scout bee mechanism are proposed in CoABCMiner to achieve cooperative learning of different rules belonging to different classes. We compared the results of CoABCMiner with several state-of-the-art algorithms using 14 benchmark data sets. Non parametric statistical tests, such as Friedman test, post hoc test, and contrast estimation based on medians are performed. Nonparametric tests determine the similarity of control algorithm among other algorithms on multiple problems. Sensitivity analysis of CoABCMiner is conducted. It is concluded that CoABCMiner can be used to discover classification rules for the data sets used in experiments, efficiently.

An algorithm evaluation for discovering classification rules with gene expression programming

AbstractIn recent years, evolutionary algorithms have been used for classification tasks. However, only a limited number of comparisons exist between classification genetic rule-based systems and gene expression programming rule-based systems. In this paper, a new algorithm for classification using gene expression programming is proposed to accomplish this task, which was compared with several classical state-of-the-art rule-based classifiers. The proposed classifier uses a Michigan approach; the evolutionary process with elitism is guided by a token competition that improves the exploration of fitness surface. Individuals that cover instances, covered previously by others individuals, are penalized. The fitness function is constructed by the multiplying three factors: sensibility, specificity and simplicity. The classifier was constructed as a decision list, sorted by the positive predictive value. The most numerous class was used as the default class. Until now, only numerical attributes are allowed and...

Predicting Students Final GPA Using Decision Trees: A Case Study

 Abstract—Educational data mining is the process of applying data mining tools and techniques to analyze data at educational institutions. In this paper, we used educational data mining to predict students' final GPA based on their grades in previous courses. In our case study, we collected students' transcript data that included their final GPA and their grades in all courses. After pre-processing the data, we applied the J48 decision tree algorithm to discover classification rules. We extracted useful knowledge for final GPA, and identify the most important courses in the students' study plan based on their grades in the mandatory courses.

Discovery of Classification Rules Using Distributed Genetic Algorithm

This paper presents a distributed genetic algorithm for the discovery of classification rules. Population is contained in the form of interconnected demes. The local selection and reproduction mechanism is used to evolve the species within demes, and diversity is enhanced by migrating rules among some of the selected demes. Subsumption operator has been finally applied to reduce the complexity of the rule set discovered. The effectiveness of the proposed distributed genetic algorithm for discovering classification rules is evaluated by comparing the results with traditional crowding GA on 10 datasets from the UCI and KEEL repository. The results confirm that the distributed GA discover classification rules with significantly higher predictive accuracy. The influence of migration operator is also analysed with respect to migration rate.

Discovering Classification Rules and Providing Ranking among Labels to Multi-Target Data Using Novel Approach

Learning from multi-label and multi-target data is always more difficult than single-label data. The complexity is incurred by multiple labels assigned to a sample as well as to provide ranking among labels. The learning algorithms used for single-label data are not suitable to learn from multi-label and multi-target data. There are many applications where multi-label and multi-target data are present, for example, image classification, gene classification, document classification and so on. Multi-label classification can be mainly classified into two methods: Problem Transformation (PT) and Algorithm Adaptation. This paper focuses on classification of multi target data by applying Pruned Set (PS) method which belongs to PT category. The PS method is capable of modelling label correlations directly and reduces over-fitting of data by pruning and sub-sampling label sets. The objective of this paper is to provide classification rules in multi-label form, provide confusion matrix which shows overall accuracy as well as label-wise accuracy for each different value of every class-label and to provide label wise ranking among class-labels for multi-target data. Experiments are performed on real multi-target datasets and three measures like Hamming loss, exact match and accuracy are compared with different PT methods. Experimental comparison shows that the PS method gives better overall results when classifying multi-target data as compared to standard PT methods.

A Genetic Algorithm Approach using Improved Fitness Function for Classification Rule Mining

rule mining from huge amount of data is a challenging issue in data mining. Classification rules describe the relationship between predicting attributes and class label attribute and thus assign class label to unseen predicting attribute values. In this paper, a Genetic algorithm approach with modified fitness function for discovering classification rules has been presented. A flexible encoding scheme for representing a rule, genetic operators like crossover, mutation and also the stated fitness function with confidence, coverage, simplicity and interestingness properties have been exploited for discovering accurate, comprehensible and interesting rules. The results of proposed Genetic algorithm have been compared with existing J48, Jrip, Naive Bayesian algorithms. Experimental results endorse that the proposed algorithm produces relatively less number of classification rules with satisfactory accuracy rates.

A Novel Method For Extracting Classification Rules Based On Ant-miner

In this paper, we propose a new method for data mining based on Ant Colony Optimization (ACO). The ACO is a metheuristic inspired by the behavior of real ants, where they search for optimal solutions by considering both local heuristic and previous knowledge, observed by pheromone changes. Good results of the ant colony algorithms for solving combinatorial optimization problems suggest an appropriate effectiveness of the approach also in the task of discovering classification rules in data mining. Mining classification rules is an important research area in data mining. Ant-Miner is an Ant Colony Optimization algorithm for classification task. This paper proposes an improved version of Ant-Miner named Ant-Miner4, which is based on Ant-Miner3 By changing the heuristic function used in the Ant-Miner3, and implementing it based on correction function of Laplace, we tried to redesign Ant-Miner to gain rules with high predictive accuracy. We compared Ant-Miner4 with the previous version (Ant-Miner3) using four data sets. The results indicated that the accuracy of the rules discovered by the new version was higher than the ones gained by the previous version.

Learning Bayesian network classifiers using ant colony optimization

Bayesian networks are knowledge representation tools that model the (in)dependency relationships among variables for probabilistic reasoning. Classification with Bayesian networks aims to compute the class with the highest probability given a case. This special kind is referred to as Bayesian network classifiers. Since learning the Bayesian network structure from a dataset can be viewed as an optimization problem, heuristic search algorithms may be applied to build high-quality networks in medium- or large-scale problems, as exhaustive search is often feasible only for small problems. In this paper, we present our new algorithm, ABC-Miner, and propose several extensions to it. ABC-Miner uses ant colony optimization for learning the structure of Bayesian network classifiers. We report extended computational results comparing the performance of our algorithm with eight other classification algorithms, namely six variations of well-known Bayesian network classifiers, cAnt-Miner for discovering classification rules and a support vector machine algorithm.

Transformer Fault Diagnosis Method Based on Rough Set and Generalized Distribution Table

Transformers are considered as significant equipments in electrical power systems, once failure ,the eco- nomic operation will be lost. To overcome this difficulty and to maintain economic operation of facilities, diverse diagnosis methods are developed to implement fault forecasting. According to intelligent complementary ideas, a fault diagnosis is proposed when there is a missing failure symptom of transformer. The core of the proposed ap- proach is a soft hybrid induction system called the Generalized Distribution Table and Rough Set System (GDT-RS) to discover classification rules. The system is based on a combination of Generalized Distribution Table (GDT) and the Rough Set methodologies. The proposed approach is applied into transformer fault diagnosis and the results indicate that it is very effective and accurate.

A Genetic Algorithm for Discovering Classification Rules in Data Mining

mining has as goal to discover knowledge from huge volume of data. Rule mining is considered as one of the usable mining method in order to obtain valuable knowledge from stored data on database systems. In this paper, a genetic algorithm-based approach for mining classification rules from large database is presented. For emphasizing on accuracy, coverage and comprehensibility of the rules and simplifying the implementation of a genetic algorithm. The design of encoding, genetic operators and fitness function of genetic algorithm for this task are discussed. Experimental results show that genetic algorithm proposed in this paper is suitable for classification rule mining and those rules discovered by the algorithm have higher classification performance to unknown data.

A framework for data mining on combinatorial game theory

Combinatorics is the study of discrete, finite spaces. Combinatorial games are games that can be studied through the use of combinatorics. They are typically, but not necessarily, two-player games with a finite set of possible states and a well-defined winning condition. The search space in combinatorial games is typically very large. In this paper, we proposed a framework to apply data mining techniques such as Bayesian classification to the combinatorial game theory, in particular, a game called Audacity. Our experimental results show that the Bayesian classification is effective for discovering classification rules in combinatorial games, such as the Audacity game.