Michigan-style Learning Classifier Systems Research Articles

Evidential Supervised Classifier System: A New Learning Classifier System Dealing with Imperfect Information

Learning Classifier Systems (LCSs) are a kind of evolutionary machine learning algorithms that provide highly adaptive components to deal with real world problems. They have been widely used in resolving complex problems such as decision making and classification. LCSs are flexible algorithms that are able to construct, incrementally, a set of rules and evolve them through the Evolutionary Algorithm (EA). Despite their efficiency, LCSs are not capable of handling imperfect information, which may lead to reduced performance in terms of classification accuracy. We propose a new accuracy-based Michigan-style LCS that integrates the belief function theory in the supervised classifier system. The belief function or evidence theory represents an efficient framework for treating imperfect information. The new approach shows promising results in real world classification problems.

Customer satisfaction prediction with Michigan-style learning classifier system

Many different classification algorithms can be use in order to analyze, classify and predict data. Learning classifier system (LCS) which is known as a genetic base machine learning system, combines the machine learning with evolutionary computing and other heuristics to produce an adaptive system that learns to solve a particular problem. This paper uses the Michigan style LCS, in the context of bank customer satisfaction to classify customers into two different groups: unsatisfied/satisfied customers. Three different Rule Compaction strategies are used to compare the rule population’s accuracy and micro/macro population size. The result specifies features that mostly influence prediction.

Implications of the curse of dimensionality for supervised learning classifier systems: theoretical and empirical analyses

Learning classifier systems are leading evolutionary machine learning systems that employ genetic algorithms to search for a set of optimally general and correct classification rules for a variety of machine learning problems, including supervised classification data mining problems. The curse of dimensionality is a phenomenon that arises when analysing data in high-dimensional spaces. Performance issues when dealing with increasing dimensionality in the training data, such as poor classification accuracy and stalled genetic search, are well known for learning classifier systems. However, a systematic study to establish the relationship between increasing dimensionality and learning challenges in these systems is lacking. The aim of this paper is to analyse the behaviour of Michigan-style learning classifier systems that use the most commonly adopted and expressive interval-based rules representation, under curse of dimensionality (also known as Hughes Phenomenon) problem. In this paper, we use well-established and mathematically founded formal geometrical properties of high-dimensional data spaces and generalisation theory of these systems to propose a formulation of such relationship. The proposed formulations are validated experimentally using a set of synthetic, two-class classification problems. The findings demonstrate that the curse of dimensionality occurs for as few as ten dimensions and negatively affects the evolutionary search with a hyper-rectangular rule representation. A number of approaches to overcome some of the difficulties uncovered by the presented analysis are then discussed. Three approaches are then analysed in more detail using a set of synthetic, two-class classification problems. Experimental study demonstrates the effectiveness of these approaches to handle increasing dimensional data.

Rule-based machine learning classification and knowledge discovery for complex problems

Learning classifier systems (LCSs) are an advantageous, powerful, and flexible class of algorithms that have, to date, been underutilized largely due to the perception that they are difficult to apply, evaluate, and interpret. ExSTraCS is an Ex tended S upervised Tra cking and C lassifying S ystem based on the Michigan-Style LCS architecture [4]. It offers an accessible, user friendly LCS platform for supervised rule-based machine learning, classification, data mining, prediction, and knowledge discovery. ExSTraCS seeks to make no assumptions about the data, and is therefore model free and particularly well suited to complex problems that are multi-factorial, interacting (non-linear), heterogeneous, noisy, class imbalanced, multi-class, or larger-scale. ExSTraCS is written in Python, open source, well documented, and freely available at sourceforge.net.

ExSTraCS 2.0: Description and Evaluation of a Scalable Learning Classifier System.

Algorithmic scalability is a major concern for any machine learning strategy in this age of 'big data'. A large number of potentially predictive attributes is emblematic of problems in bioinformatics, genetic epidemiology, and many other fields. Previously, ExS-TraCS was introduced as an extended Michigan-style supervised learning classifier system that combined a set of powerful heuristics to successfully tackle the challenges of classification, prediction, and knowledge discovery in complex, noisy, and heterogeneous problem domains. While Michigan-style learning classifier systems are powerful and flexible learners, they are not considered to be particularly scalable. For the first time, this paper presents a complete description of the ExS-TraCS algorithm and introduces an effective strategy to dramatically improve learning classifier system scalability. ExSTraCS 2.0 addresses scalability with (1) a rule specificity limit, (2) new approaches to expert knowledge guided covering and mutation mechanisms, and (3) the implementation and utilization of the TuRF algorithm for improving the quality of expert knowledge discovery in larger datasets. Performance over a complex spectrum of simulated genetic datasets demonstrated that these new mechanisms dramatically improve nearly every performance metric on datasets with 20 attributes and made it possible for ExSTraCS to reliably scale up to perform on related 200 and 2000-attribute datasets. ExSTraCS 2.0 was also able to reliably solve the 6, 11, 20, 37, 70, and 135 multiplexer problems, and did so in similar or fewer learning iterations than previously reported, with smaller finite training sets, and without using building blocks discovered from simpler multiplexer problems. Furthermore, ExS-TraCS usability was made simpler through the elimination of previously critical run parameters.

An Analysis Pipeline with Statistical and Visualization-Guided Knowledge Discovery for Michigan-Style Learning Classifier Systems.

Michigan-style learning classifier systems (M-LCSs) represent an adaptive and powerful class of evolutionary algorithms which distribute the learned solution over a sizable population of rules. However their application to complex real world data mining problems, such as genetic association studies, has been limited. Traditional knowledge discovery strategies for M-LCS rule populations involve sorting and manual rule inspection. While this approach may be sufficient for simpler problems, the confounding influence of noise and the need to discriminate between predictive and non-predictive attributes calls for additional strategies. Additionally, tests of significance must be adapted to M-LCS analyses in order to make them a viable option within fields that require such analyses to assess confidence. In this work we introduce an M-LCS analysis pipeline that combines uniquely applied visualizations with objective statistical evaluation for the identification of predictive attributes, and reliable rule generalizations in noisy single-step data mining problems. This work considers an alternative paradigm for knowledge discovery in M-LCSs, shifting the focus from individual rules to a global, population-wide perspective. We demonstrate the efficacy of this pipeline applied to the identification of epistasis (i.e., attribute interaction) and heterogeneity in noisy simulated genetic association data.

Facetwise Analysis of XCS for Problems With Class Imbalances

Michigan-style learning classifier systems (LCSs) are online machine learning techniques that incrementally evolve distributed subsolutions which individually solve a portion of the problem space. As in many machine learning systems, extracting accurate models from problems with class imbalances-that is, problems in which one of the classes is poorly represented with respect to the other classes-has been identified as a key challenge to LCSs. Empirical studies have shown that Michigan-style LCSs fail to provide accurate subsolutions that represent the minority class in domains with moderate and large disproportion of examples per class; however, the causes of this failure have not been analyzed in detail. Therefore, the aim of this paper is to carefully examine the effect of class imbalances on different LCS components. The analysis focuses on XCS, which is the most-relevant Michigan-style LCS, although the models could be easily adapted to other LCSs. Design decomposition is used to identify five elements that are crucial to guaranteeing the success of LCSs in domains with class imbalances, and facetwise models that explain these different elements for XCS are developed. All theoretical models are validated with artificial problems. The integration of all these models enables us to identify the sweet spot where XCS is able to scalably and efficiently evolve accurate models of rare classes; furthermore, facetwise analysis is used as a tool for designing a set of configuration guidelines that have to be followed to ensure convergence. When properly configured, XCS is shown to be able to solve highly unbalanced problems that previously eluded solution.

Problem solution sustenance in XCS: Markov chain analysis of niche support distributions and the impact on computational complexity

Michigan-style learning classifier systems iteratively evolve a distributed solution to a problem in the form of potentially overlapping subsolutions. Each problem niche is covered by subsolutions that are represented by a set of predictive rules, termed classifiers. The genetic algorithm is designed to evolve classifier structures that together cover the whole problem space and represent a complete problem solution. An obvious challenge for such an online evolving, distributed knowledge representation is to continuously sustain all problem subsolutions covering all problem niches, that is, to ensure niche support. Effective niche support depends both on the probability of reproduction and on the probability of deletion of classifiers in a niche. In XCS, reproduction is occurrence-based whereas deletion is support-based. In combination, niche support is assured effectively. In this paper we present a Markov chain analysis of the niche support in XCS, which we validate experimentally. Evaluations in diverse Boolean function settings, which require non-overlapping and overlapping solution structures, support the theoretical derivations. We also consider the effects of mutation and crossover on niche support. With respect to computational complexity, the paper shows that XCS is able to maintain (partially overlapping) niches with a computational effort that is linear in the inverse of the niche occurrence frequency.