Rule-based Classification Model Research Articles

Strength-based learning classifier systems revisited: Effective rule evolution in supervised classification tasks

Learning classifier systems (LCS) are machine learning systems designed to work for both multi-step and single-step decision tasks. The latter case presents an interesting challenge for such algorithms, especially when they are applied to real-world data mining (DM) problems. The present investigation departs from the popular approach of applying accuracy-based LCS to single-step classification and aims to uncover the potential of strength-based LCS in such tasks. Although the latter family of algorithms have often been associated with poor generalization and performance, we aim at alleviating these problems by defining appropriate extensions to the traditional strength-based LCS framework. These extensions are detailed and their effect on system performance is studied through the application of the proposed algorithm on a set of artificial problems, designed to challenge its scalability and generalization abilities. The comparison of the proposed algorithm with UCS, its state-of-the-art accuracy-based counterpart, emphasizes the effects of our extended strength-based approach and validates its competitiveness in multi-class problems with various class distributions. Overall, our work presents an investigation of strength-based LCS in the domain of supervised classification. Our extensive analysis of the learning dynamics involved in these systems provides proof of their potential as real-world DM tools, inducing tractable rule-based classification models, even in the presence of severe class imbalances.

ApicoAP: The First Computational Model for Identifying Apicoplast-Targeted Proteins in Multiple Species of Apicomplexa

BackgroundMost of the parasites of the phylum Apicomplexa contain a relict prokaryotic-derived plastid called the apicoplast. This organelle is important not only for the survival of the parasite, but its unique properties make it an ideal drug target. The majority of apicoplast-associated proteins are nuclear encoded and targeted post-translationally to the organellar lumen via a bipartite signaling mechanism that requires an N-terminal signal and transit peptide (TP). Attempts to define a consensus motif that universally identifies apicoplast TPs have failed.Methodology/Principal FindingsIn this study, we propose a generalized rule-based classification model to identify apicoplast-targeted proteins (ApicoTPs) that use a bipartite signaling mechanism. Given a training set specific to an organism, this model, called ApicoAP, incorporates a procedure based on a genetic algorithm to tailor a discriminating rule that exploits the known characteristics of ApicoTPs. Performance of ApicoAP is evaluated for four labeled datasets of Plasmodium falciparum, Plasmodium yoelii, Babesia bovis, and Toxoplasma gondii proteins. ApicoAP improves the classification accuracy of the published dataset for P. falciparum to 94%, originally 90% using PlasmoAP.Conclusions/SignificanceWe present a parametric model for ApicoTPs and a procedure to optimize the model parameters for a given training set. A major asset of this model is that it is customizable to different parasite genomes. The ApicoAP prediction software is available at http://code.google.com/p/apicoap/ and http://bcb.eecs.wsu.edu.

Effective search for Pittsburgh learning classifier systems via estimation of distribution algorithms

Pittsburgh-style learning classifier systems (LCSs), in which an entire candidate solution is represented as a set of variable number of rules, combine supervised learning with genetic algorithms (GAs) to evolve rule-based classification models. It has been shown that standard crossover operators in GAs do not guarantee an effective evolutionary search in many sophisticated problems that contain strong interactions between features. In this paper, we propose a Pittsburgh-style learning classifier system based on the Bayesian optimization algorithm with the aim of improving the effectiveness and efficiency of the rule structure exploration. In the proposed method, classifiers are generated and recombined at two levels. At the lower level, single rules contained in classifiers are produced by sampling Bayesian networks which characterize the global statistical information extracted from the current promising rules in the search space. At the higher level, classifiers are recombined by rule-wise uniform crossover operators to keep the semantics of rules in each classifier. Experimental studies on both artificial and real world binary classification problems show that the proposed method converges faster while achieving solutions with the same or even higher accuracy compared with the original Pittsburgh-style LCSs.

A Fuzzy Association Rule-Based Classification Model for High-Dimensional Problems With Genetic Rule Selection and Lateral Tuning

The inductive learning of fuzzy rule-based classification systems suffers from exponential growth of the fuzzy rule search space when the number of patterns and/or variables becomes high. This growth makes the learning process more difficult and, in most cases, it leads to problems of scalability (in terms of the time and memory consumed) and/or complexity (with respect to the number of rules obtained and the number of variables included in each rule). In this paper, we propose a fuzzy association rule-based classification method for high-dimensional problems, which is based on three stages to obtain an accurate and compact fuzzy rule-based classifier with a low computational cost. This method limits the order of the associations in the association rule extraction and considers the use of subgroup discovery, which is based on an improved weighted relative accuracy measure to preselect the most interesting rules before a genetic postprocessing process for rule selection and parameter tuning. The results that are obtained more than 26 real-world datasets of different sizes and with different numbers of variables demonstrate the effectiveness of the proposed approach.

Development of an automated physical activity classification application for mobile phones.

Physical activity classification is an objective approach to assess levels of physical activity, and indicates an individual's degree of functional ability. It is significant for a number of the disciplines, such as behavioural sciences, physiotherapy, etc. Accelerometry is found to be a practical and low cost method for activity classification that could provide an objective and efficient measurement of people's daily activities. This paper utilises a mobile phone with a built-in tri-axial accelerometer sensor to automatically classify normal physical activities. A rule-based activity classification model, which can recognise 4 common daily activities (lying, walking, sitting, and standing) and 6 transitions between postural orientations, is introduced here. In this model, three types of statuses (walking/ transition, lying, and sitting/standing) are first classified based on the kinetic energy and upright angle. Transitions are then separated from walking and assigned to the corresponding type using upright angle algorithm. To evaluate the performance of this developed application, a trial is designed with 8 healthy adult subjects, who are required to perform a 6-minute activity routine with an iPhone fixed at the waist position. Based on the evaluation result, our application measures the length of time of each activity accurately and the achieved sensitivity of each activity classification exceeds 90% while the achieved specificity exceeds 96%. Meanwhile, regarding the transition identification, the sensitivities are high in stand-to-sit (80%) and low in sit-to-stand (56%).

Genetic-based machine learning systems are competitive for pattern recognition

During the last decade, research on Genetic-Based Machine Learning has resulted in several proposals of supervised learning methodologies that use evolutionary algorithms to evolve rule-based classification models. Usually, these new GBML approaches are accompanied by little experimentation and there is a lack of comparisons among different proposals. Besides, the competitiveness of GBML systems with respect to non-evolutionary, highly-used machine learning techniques has only been partially studied. This paper reviews the state of the art in GBML, selects some of the best representatives of different families, and compares the accuracy and the interpretability of their models. The paper also analyzes the behavior of the GBML approaches with respect to some of the most influential machine learning techniques that belong to different learning paradigms such as decision trees, support vector machines, instance-based classifiers, and probabilistic classifiers. The experimental observations emphasize the suitability of GBML systems for performing classification tasks. Moreover, the analysis points out the strengths of the different systems, which can be used as recommendation guidelines on which systems should be employed depending on whether the user prefers to maximize the accuracy or the interpretability of the models.

A RULE-BASED SOFTWARE QUALITY CLASSIFICATION MODEL

A rule-based classification model is presented to identify high-risk software modules. It utilizes the power of rough set theory to reduce the number of attributes, and the equal frequency binning algorithm to partition the values of the attributes. As a result, a set of conjuncted Boolean predicates are formed. The model is inherently influenced by the practical needs of the system being modeled, thus allowing the analyst to determine which rules are to be used for classifying the fault-prone and not fault-prone modules. The proposed model also enables the analyst to control the number of rules that constitute the model. Empirical validation of the model is accomplished through a case study of a large legacy telecommunications system. The ease of rule interpretation and the transparency of the functional aspects of the model are clearly demonstrated. It is concluded that the new model is effective in achieving the software quality classification.

From Fuzzy Clustering to a Fuzzy Rule-Based Fault Classification Model

The article overviews current trends in research studies related to reliability prediction and prognostics. The trends are organized into three major types of prognostic models: failure data models, stressor models, and degradation models. Methods in each of these categories are presented and examples are given. Additionally, three particular computational prognostic approaches are considered; these are Markov chain-based models, general path models, and shock models. A Bayesian technique is then presented which integrates the prognostic types by incorporate prior reliability knowledge into the prognostic models. Finally, the article also discusses the usage of diagnostic/prognostic predictions for optimal control.

From Fuzzy Clustering to a Fuzzy Rule-Based Classification Model

From Fuzzy Clustering to a Fuzzy Rule-Based Classification Model

Fuzzy classification of microbial biomass and enzyme activities in grassland soils

Soil microbial properties are widely used as indicators of soil quality. The interpretation of soil microbial processes, however, is difficult because of their regional and seasonal heterogeneity as well as the lack of reference values. One possibility to overcome these limitations to apply the fuzzy set theory. This approach more realistically describes ecological systems because it considers natural ambiguity and complexity. The present study applies a fuzzy rule-based classification model to define soil quality based on soil microbial biomass, N-mineralisation, enzyme activity data (urease, xylanase, phosphatase, arylsulfatase) and soil organic matter. The data have been collected from different grassland sites in the European Union over a period of 20 years. The fuzzy model is based on a rule system derived from a training set using simulated annealing as an optimisation algorithm. For each variable, nine triangular fuzzy sets were defined for consideration as possible rule arguments. The model uses the t-norm for combination of arguments, product inference, the weighted sum as rule combination and the maximum method for defuzzification. The output is the assignment of membership of the object to a given soil quality class. The soil quality classes (very poor, poor, medium, high, very high) were defined by five heavy metal contamination levels (very high, high, medium, low, no). A predefined number of fuzzy rules were assessed using a simulated annealing algorithm. The fuzzy model was validated by a test file by assigning the soils to the quality class with the highest response value. The fuzzy model yielded an overall coincidence of 88.8% between observed and simulated results. The most sensitive index of soil quality was microbial biomass. N-mineralisation was a good indicator for the high-quality soils, while urease and arylsulfatase were important indicators for heavily contaminated, poor soil quality. Xylanase and phosphatase behaved ambivalently. Including soil organic carbon in the model decreased its effectiveness by 6.5%. We suggest that the presented fuzzy model based on soil microbial properties could be applied not only to soil degradation, upscaling and prediction, but also to judge the response of soils to environmental changes.

Resource oriented selection of rule-based classification models: An empirical case study

The amount of resources allocated for software quality improvements is often not enough to achieve the desired software quality. Software quality classification models that yield a risk-based quality estimation of program modules, such as fault-prone (fp) and not fault-prone (nfp), are useful as software quality assurance techniques. Their usefulness is largely dependent on whether enough resources are available for inspecting the fp modules. Since a given development project has its own budget and time limitations, a resource-based software quality improvement seems more appropriate for achieving its quality goals. A classification model should provide quality improvement guidance so as to maximize resource-utilization. We present a procedure for building software quality classification models from the limited resources perspective. The essence of the procedure is the use of our recently proposed Modified Expected Cost of Misclassification (MECM) measure for developing resource-oriented software quality classification models. The measure penalizes a model, in terms of costs of misclassifications, if the model predicts more number of fp modules than the number that can be inspected with the allotted resources. Our analysis is presented in the context of our Rule-Based Classification Modeling (RBCM) technique. An empirical case study of a large-scale software system demonstrates the promising results of using the MECM measure to select an appropriate resource-based rule-based classification model.

Robust rule-based prediction

This paper studies a problem of robust rule-based classification, i.e., making predictions in the presence of missing values in data. This study differs from other missing value handling research in that it does not handle missing values but builds a rule-based classification model to tolerate missing values. Based on a commonly used rule-based classification model, we characterize the robustness of a hierarchy of rule sets as k-optimal rule sets with the decreasing size corresponding to the decreasing robustness. We build classifiers based on k-optimal rule sets and show experimentally that they are more robust than some benchmark rule-based classifiers, such as C4.5rules and CBA. We also show that the proposed approach is better than two well-known missing value handling methods for missing values in test data.

An association rule mining-based methodology for automated detection of ischemic ECG beats

Currently, an automated methodology based on association rules is presented for the detection of ischemic beats in long duration electrocardiographic (ECG) recordings. The proposed approach consists of three stages. 1) Preprocessing: Noise is removed and all the necessary ECG features are extracted. 2) Discretization: The continuous valued features are transformed to categorical. 3) CLASSIFICATION: An association rule extraction algorithm is utilized and a rule-based classification model is created. According to the proposed methodology, electrocardiogram (ECG) features extracted from the ST segment and the T-wave, as well as the patient's age, were used as inputs. The output was the classification of the beat as ischemic or not. Various algorithms were tested both for discretization and for classification using association rules. To evaluate the methodology, a cardiac beat dataset was constructed using several recordings of the European Society of Cardiology ST-T database. The obtained sensitivity (Se) and specificity (Sp) was 87% and 93%, respectively. The proposed methodology combines high accuracy with the ability to provide interpretation for the decisions made, since it is based on a set of association rules.

Detecting noisy instances with the rule-based classification model

The performance of a classification model is invariably affected by the characteristics of measurement data it is built upon. If quality of the data is generally poor, then the classification model will demonstrate poor performance. The amount of noisy instances present in a given dataset is a good reflection of quality of the data. The detection and removal of noisy data instances will improve quality of the data, and consequently the performance of the classification model. This study presents an attractive and user-friendly approach for detecting data noise based on Boolean rules generated from the measurement data. The approach follows a simple and replicable approach that analyzes the rules to detect mislabeled noisy instances in the training dataset. Such instances are treated as data noise, and are removed to obtain a clean dataset. A case study of a software measurement dataset with known noisy instances is used to demonstrate the effectiveness of our approach. The dataset is obtained from a NASA software project developed for realtime predictions based on simulations. It is empirically demonstrated that the proposed approach is extremely effective in detecting noise in the dataset; in fact, the approach detected 100% of the known noisy instances. The proposed approach is compared with noise filtering based on five classification filters and an ensemble filter of five classifiers. We also demonstrate that the proposed approach shows excellent promise in detecting noisy instances in several (six) independent and real-world software measurement datasets with unknown noisy instances.

Determining the significance of threshold values uncertainty in rule-based classification models

Ecological models not only represent a tool for studying ecosystems, but also function as a key source for informing environmental policy. As such, these models are expected to present and convey information and uncertainty as accurately as possible. As the complexity of these models increases, the resultant output also becomes more complex and difficult to interpret. However, understanding model output is not limited to interpreting complex dynamics; scientists must also contend with the possibility of model error and uncertainty. It is the uncertainty associated with the output that provides information about the reliability of model predictions, and therefore plays a pivotal role in model evaluation and interpretation. Despite its importance, uncertainty frequently is ignored. We present a methodology to determine the significance of uncertainty in rule-based classification models. This is a stepwise methodology that, using a genetic algorithm, determines the effects of uncertainty in model output and identifies the most likely alternative results. It also computes a measure of confidence to help the modeler evaluate how the predictions are affected by uncertainties. We present a case study applying the methodology to the global vegetation model Mapped Atmosphere-Plant-Soil System (MAPSS) where the measure of confidence contains information about the dissimilarity between alternative outcome classes, and their spatial configuration on the landscape. Fuzzy sets are used to portray most likely alternative results as integral part of model output in a spatially explicit format.

Using Coverage as a Model Building Constraint in Learning Classifier Systems

Promoting and maintaining diversity is a critical requirement of search in learning classifier systems (LCSs). What is required of the genetic algorithm (GA) in an LCS context is not convergence to a single global maximum, as in the standard optimization framework, but instead the generation of individuals (i.e., rules) that collectively cover the overall problem space. COGIN (COverage-based Genetic INduction) is a system designed to exploit genetic recombination for the purpose of constructing rule-based classification models from examples. The distinguishing characteristic of COGIN is its use of coverage of training set examples as an explicit constraint on the search, which acts to promote appropriate diversity in the population of rules over time. By treating training examples as limited resources, COGIN creates an ecological model that simultaneously accommodates a dynamic range of niches while encouraging superior individuals within a niche, leading to concise and accurate decision models. Previous experimental studies with COGIN have demonstrated its performance advantages over several well-known symbolic induction approaches. In this paper, we examine the effects of two modifications to the original system configuration, each designed to inject additional diversity into the search: increasing the carrying capacity of training set examples (i.e., increasing coverage redundancy) and increasing the level of disruption in the recombination operator used to generate new rules. Experimental results are given that show both types of modifications to yield substantial improvements to previously published results.