XCS Classifier System Research Articles

Mechanisms to Alleviate Over-Generalization in XCS for Continuous-Valued Input Spaces

In the field of rule-based approaches to Machine Learning, the XCS classifier system (XCS) is a well-known representative of the learning classifier systems family. By using a genetic algorithm (GA), the XCS aims at forming rules or so-called classifiers which are as general as possible to achieve an optimal performance level. A too high generalization pressure may lead to over-general classifiers degrading the performance of XCS. To date, no method exists for XCS for real-valued input spaces (XCSR) and XCS for function approximation (XCSF) to handle over-general classifiers ensuring an accurate population. The Absumption mechanism and the Specify operator, both developed for XCS with binary inputs, provide a promising basis for over-generality handling in XCSR and XCSF. This paper introduces adapted versions of Absumption and Specify by proposing different identification and specialization strategies for the application in XCSR and XCSF. To determine their potential, the adapted techniques are evaluated in different classification problems, i.e., common benchmarks and real-world data from the agricultural domain, in a multi-step problem as well as different regression tasks. Our experimental results show that the application of these techniques leads to significant improvements of the accuracy of the generated classifier population in the applied benchmarks, data sets, multi-step problems and regression tasks, especially when they tend to form over-general classifiers. Furthermore, considering the working principle of the proposed techniques, the intended decrease in overall classifier generality can be confirmed.

Learning Regular Expressions Using XCS-Based Classifier System

Evolutionary machine learning research aims to develop classifier systems that can solve complex and hard tasks. This paper addresses the problem of inferring a regular expression from a given set of strings for automating the task of information extraction. To the best of our knowledge, this paper is the first to propose the extension of accuracy-based classifier system XCS to learn the regular expressions for text extraction. This new system named as XCSREA includes tree-like code fragments to learn regular expressions. The genetic algorithm in action sets uses two-point crossover with uniform mutation and Roulette wheel parent selection method. Seven different datasets, each with three different lengths, are used to compare the performance of the proposed model with standard genetic programming (GP) approach. The experimental results demonstrate that XCSREA outperforms standard GP approach when sufficiently large numbers of classifiers are used.

An Analysis of Rule Deletion Scheme in XCS on Reinforcement Learning Problem

The XCS classifier system is an evolutionary rule-based learning technique powered by a Q-learning like learning mechanism. It employs a global deletion scheme to delete rules from all rules covering all state-action pairs. However, the optimality of this scheme remains unclear owing to the lack of intensive analysis. We here introduce two deletion schemes: 1) local deletion, which can be applied to a subset of rules covering each state (a match set), and 2) stronger local deletion, which can be applied to a more specific subset covering each state-action pair (an action set). The aim of this paper is to reveal how the above three deletion schemes affect the performance of XCS. Our analysis shows that the local deletion schemes promote the elimination of inaccurate rules compared with the global deletion scheme. However, the stronger local deletion scheme occasionally deletes a good rule. We further show that the two local deletion schemes greatly improve the performance of XCS on a set of noisy maze problems. Although the localization strength of the proposed deletion schemes may require consideration, they can be adequate for XCS rather than the original global deletion scheme.

Rule reduction by selection strategy in XCS with adaptive action map

The XCS classifier system is a rule-based evolutionary machine learning system. XCS evolves classifiers in order to learn generalized solutions. The XCS with adaptive action mapping (XCSAM) is inherited from XCS, which evolves a best action map where it evolves classifiers that advocate the best action in every state. Accordingly, XCSAM can potentially evolve solutions that are more compact than XCS, which in contrast focuses on a complete action map. Previous experimental results however have shown that, in some problems, XCSAM may produce solutions with more classifiers than XCS. In this paper, we initially show that the original fitness-based selection strategy of XCS produces non effective classifiers which are not likely to be in the best action map (i.e., they are inaccurate ones or do not have best actions) in XCSAM. Then, we introduce a new selection strategy for XCSAM that promotes the evolution of classifiers advocating the best action map and thus produces more compact solutions. The new strategy selects classifiers based both on their fitness (like XCS) and on the parameter optimality of action of XCSAM. The result is a pressure towards classifiers that are accurate and advocate the best actions. We present analyses showing that the new selection strategy successfully enables XCSAM to focus on classifiers having best actions. Our experimental results show that XCSAM with the new selection strategy (called XCSAM-SS) can evolve smaller solutions than XCS (and the original XCSAM) both in single-step and multi-step problems. As a consequence, XCSAM can also learn with smaller iterations than XCS in the single-step problem. Our conclusion is that, as the best action map potentially has a compact solution, XCSAM evolves a much compact solution than XCS by adding an adequate selection strategy.

XCS-SL: a rule-based genetic learning system for sequence labeling

Sequence labeling is an interesting classification domain where, like normal classification, every input has a class label, but unlike normal classification, prediction of an input’s label may depend on the values of other inputs or their classes, and so a learner may need to refer to inputs and classes at different time stamps to classify the current input. This is more difficult because a learner does not know where and how many inputs are needed to classify the current input. Our interest is in learning general rules for sequence labeling. The XCS algorithm is a rule-based knowledge discovery system powered by a genetic algorithm which has often been used for classification. Here we present XCS-SL, an extension of XCS classifier system which can be applicable to sequence labeling. Towards an application of Learning Classifier System (LCS) to sequence labeling, we propose a new classifier condition with memory (called a variable-length condition) and a rule-discovery system for the new classifier condition, which enables XCS to apply it to sequence labeling. In XCS-SL, classification rules (called “classifiers” here) can include extra conditions on previous inputs, which act as memories. In sequence labeling, the number of conditions/memories needed may be different for each input, hence, using a fixed number of conditions (i.e., fixed-length condition) for all classifiers is not a good solution. Instead, XCS-SL classifiers have a variable-length condition to provide more or less memory. The genetic algorithm can grow and shrink conditions to find a suitable memory size. On two synthetic benchmark problems XCS-SL learns optimal classifiers, and on a real-world sequence labeling task it derives high classification accuracy and discovers interesting knowledge that shows dependencies between inputs at different times. The comprehensively described system is the first application of a LCS to sequence labeling and we consider it a promising direction for future work.

An Expert Clinical System For Diagnosing Obstructive Sleep Apnea With Help From The Xcsr Classifier

Obstructive sleep apnea is a common condition with serious neural-psychological complications and cardiovascular problems if not diagnosed and treated in time. Despite the importance of this disease in our country, it has not received much attention and there are few centers for evaluating patients suffering from it. In this article, an intelligent method is introduced for diagnosing obstructive sleep apnea that uses features extracted from changes in heart rate and respiratory signals in the ECG as input for training and testing the modified XCS classifier system. Comparison of results obtained from implementing the mentioned method with those of other methods on physionet database showed desirable performance and high accuracy of the proposed system in diagnosing obstructive sleep

Learning classifier systems with memory condition to solve non-Markov problems

In the family of Learning Classifier Systems, the classifier system XCS has been successfully used for many applications. However, the standard XCS has no memory mechanism and can only learn optimal policy in Markov environments, where the optimal action is determined solely by the state of current sensory input. In practice, most environments are partially observable environments on agent's sensation, which are also known as non-Markov environments. Within these environments, XCS either fails, or only develops a suboptimal policy, since it has no memory. In this work, we develop a new classifier system based on XCS to tackle this problem. It adds an internal message list to XCS as the memory list to record input sensation history, and extends a small number of classifiers with memory conditions. The classifier's memory condition, as a foothold to disambiguate non-Markov states, is used to sense a specified element in the memory list. Besides, a detection method is employed to recognize non-Markov states in environments, to avoid these states controlling over classifiers' memory conditions. Furthermore, four sets of different complex maze environments have been tested by the proposed method. Experimental results show that our system is one of the best techniques to solve partially observable environments, compared with some well-known classifier systems proposed for these environments.

Knowledge extraction and rule set compaction in XCS for non-Markov multi-step problems

The application of the XCS family of classifier systems within multi-step problems has shown promise, but XCS classifier systems usually solve those problems with a large number of classifiers. If they can generate compact and readily interpretable solutions when used in multi-step problems, it will enhance their applicability and usefulness in a wide range of robot control and knowledge discovery tasks. In this paper, we describe some methods developed for XCS to compact the rule set and acquire explicit policy knowledge for multi-step problems, especially non-Markov problems. Experimental results show the ability to obtain a very compact rule set out of the final population of classifiers with little or no degradation of overall performance.

Adding memory condition to learning classifier systems to solve partially observable environments

Within the paradigm of learning classifier systems, extended classifier system XCS is outstanding. However, the original XCS has no memory mechanism and can only learn optimal policy in Markovian environments, where the optimal action is determined solely by the state of current sensory input. But in practice, most environments are partially observable environments with respect to agent's sensation, and they form the most general class of environments: non-Markov environments. In these environments, XCS either fails completely, or only develops a suboptimal policy, since it is memoryless. In this paper, we develop a new learning classifier system based on XCS, named 'XCSMM', which adds an internal message to XCS as an internal memory, and then extends the classifier with a memory condition that is used to sense the internal memory. XCSMM holds a simple and clear memory mechanism, which is easy to understand and implement. Besides, four sets of different complex maze problems have been employed to test XCSMM. Experimental results show that XCSMM is able to evolve optimal or suboptimal solutions in most non-Markovian environments.

Learning classifier system with average reward reinforcement learning

In the family of Learning Classifier Systems, the classifier system XCS is most widely used and investigated. However, the standard XCS has difficulties solving large multi-step problems, where long action chains are needed to get delayed rewards. Up to the present, the reinforcement learning technique in XCS has been based on Q-learning, which optimizes the discounted total reward received by an agent but tends to limit the length of action chains. However, there are some undiscounted reinforcement learning methods available, such as R-learning and average reward reinforcement learning in general, which optimize the average reward per time step. In this paper, R-learning is used as the reinforcement learning employed by XCS, to replace Q-learning. The modification results in a classifier system that is rapid and able to solve large maze problems. In addition, it produces uniformly spaced payoff levels, which can support long action chains and thus effectively prevent the occurrence of overgeneralization.

Evolving optimum populations with XCS classifier systems

The main goal of the research direction is to extract building blocks of knowledge from a problem domain. Once extracted successfully, these building blocks are to be used in learning more complex problems of the domain, in an effort to produce a scalable learning classifier system (LCS). However, whilst current LCS (and other evolutionary computation techniques) discover good rules, they also create sub-optimum rules. Therefore, it is difficult to separate good building blocks of information from others without extensive post-processing. In order to provide richness in the LCS alphabet, code fragments similar to tree expressions in genetic programming are adopted. The accuracy-based XCS concept is used as it aims to produce maximally general and accurate classifiers, albeit the rule base requires condensation (compaction) to remove spurious classifiers. Serendipitously, this work on scalability of LCS produces compact rule sets that can be easily converted to the optimum population. The main contribution of this work is the ability to clearly separate the optimum rules from others without the need for expensive post-processing for the first time in LCS. This paper identifies that consistency of action in rich alphabets guides LCS to optimum rule sets.

Biased trader model and analysis of financial market dynamics

This study focuses on various trader behaviors that affect market dynamics. In particular, the effects of a covering mechanism, learning mechanism and bias mechanism are analyzed through agent-based financial market model. An XCS classifier system is used to model trader learning mechanism. A trader model is proposed to formulate a trader decision model that combines bias mechanisms with learning mechanisms. The results reveal that biased traders survive under evolving markets and affect price dynamics. The model contributes to understanding the market behavior and potential sources of deviation from efficient market equilibrium.

Resource management and scalability of the XCSF learning classifier system

It has been shown many times that the evolutionary online learning XCS classifier system is a robustly generalizing reinforcement learning system, which also yields highly competitive results in data mining applications. The XCSF version of the system is a real-valued function approximation system, which learns piecewise overlapping local linear models to approximate an iteratively sampled function. While the theory on the binary domain side goes as far as showing that XCS can PAC learn a slightly restricted set of k-DNF problems, theory for XCSF is still rather sparse. This paper takes the theory from the XCS side and projects it onto the real-valued XCSF domain. For a set of functions, in which fitness guidance is given, we even show that XCSF scales optimally with respect to the population size, requiring only a constant overhead to ensure that the evolutionary process can locally optimize the evolving structures. Thus, we provide foundations concerning scalability and resource management for XCSF. Furthermore, we reveal dimensions of problem difficulty for XCSF — and local linear learners in general — showing how structural alignment, that is, alignment of XCSF’s solution representation to the problem structure, can reduce the complexity of challenging problems by orders of magnitude.

Using the XCS classifier system for portfolio allocation of MSCI index component stocks

In a recent study, Schulenburg and Ross (2001) proposed the LCS for short-term stock forecast. Studley and Bull (2007) proposed the extended classifier system (XCS) agent to model different traders by supplying different input information. Announcement made by Morgan Stanley Capital Investment (MSCI) regarding the additions, removals, and even the weights of the component stocks in its country indices every quarter generally would cause changes to the prices and/or trade volumes of the associated component stocks. This paper takes an XCS in artificial intelligence to dynamically learn and adapt to the changes to the component stocks in order to optimize portfolio allocation of the component stocks. Since these price trends of MSCI component stocks are influenced by unknown and unpredictable surroundings, using XCS to model the fluctuations on financial market allows for the capability to discover the patterns of future trends. This simulation works on the basis of the changes to 121 component stocks in the MSCI Taiwan index between 1998 and 2009 suggests the XCS can produce the great profit and optimize portfolio allocation.

SELF-ADAPTIVE LEARNING CLASSIFIER SYSTEM

This article introduces a new kind of self-adaptation in discovery mechanism of learning classifier system XCS. Unlike the previous approaches, which incorporate self-adaptive parameters in the representation of an individual, proposed model evolves competitive population of the reduced XCSs, which are able to adapt both classifiers and genetic parameters. The experimental comparisons of self-adaptive mutation rate XCS and standard XCS interacting with 11-bit, 20-bit, and 37-bit multiplexer environment were provided. It has been shown that adapting the mutation rate can give an equivalent or better performance to known good fixed parameter settings, especially for computationally complex tasks. Moreover, the self-adaptive XCS is able to solve the problem of inappropriate for a standard XCS parameters.

A Recursive Classifier System for Partially Observable Environments

Previously we introduced Parallel Specialized XCS (PSXCS), a distributed-architecture classifier system that detects aliased environmental states and assigns their handling to created subordinate XCS classifier systems. PSXCS uses a history-window approach, but with novel efficiency since the subordinateXCSs, which employ the windows, are only spawned for parts of the state space that are actually aliased. However, because the window lengths are finite and set manually, PSXCS may fail to be optimal in difficult test mazes. This paper introduces Recursive PSXCS (RPSXCS) that automatically spawns windows wherever more history is required. Experimental results show that RPSXCS is both more powerful and learns faster than PSXCS. The present research suggests new potential for history approaches to partially observable environments.

GICS-GA servicio Grid de clasificación de imágenes LANDSAT que utiliza el sistema clasificador inteligente XCS

El proyecto descrito en este artículo explora la aplicación de la tecnología Grid como apoyo a la formación y la investigación, a través de la promoción de un sistema clasificador de imágenes de satélite a un ambiente Grid, mediante la implementación de un servicio Grid al que tendrá acceso la comunidad de RENATA por medio de un portal Grid desarrollado para tal fin. El desarrollo de nuevas tecnologías de comunicación que soportan mayores anchos de banda y el advenimiento de redes de alta velocidad, han dado un gran impulso a la computación Grid, y ha incentivado la conformación de comunidades científicas que ahora pueden compartir sus recursos, aunque estos se encuentren dispersos geográficamente, recursos tales como hardware, software y capacidad de procesamiento y almacenamiento [6]. Una de las áreas que puede sacar provecho de esta nueva infraestructura es la relacionada con el procesamiento de imágenes de satélite, y más concretamente el proceso de clasificación, teniendo en cuenta la gran cantidad de recursos de procesamiento que este exige. En este documento se describen los resultados de investigación del proyecto GICS-GA que consiste en la implementación de un servicio Grid de clasificación de imágenes Landsat que implementa la funcionalidad definida por los estándares del Consorcio Abierto Geoespacial (Open Geospatial Consortium, OGC) y que para las operaciones de entrenamiento y clasificación utiliza el clasificador inteligente XCS, que al igual que la mayoría de los Sistemas Clasificadores Inteligentes hace uso de los algoritmos genéticos como mecanismo evolutivo.

Using the XCS Classifier System for Multi-objective Reinforcement Learning Problems

We investigate the performance of a learning classifier system in some simple multi-objective, multi-step maze problems, using both random and biased action-selection policies for exploration. Results show that the choice of action-selection policy can significantly affect the performance of the system in such environments. Further, this effect is directly related to population size, and we relate this finding to recent theoretical studies of learning classifier systems in single-step problems.

Evolving classifiers on field programmable gate arrays: Migrating XCS to FPGAs

The paper presents the first results of the prototype implementation of the eXtended learning Classifier System (XCS) in hardware and precisely on Field Programmable Gate Arrays. For this purpose we introduce a version of the XCS classifier system completely based on integer arithmetic, that we name XCSi, instead of the usual floating point one, to exploit the peculiarities and overcome the limitations of the hardware platform. We present an analysis of XCSi performance and the guidelines for a hardware implementation, showing that, although there is a dramatic reduction of available precision, the integer version of XCS can reach optimal performance in all problems considered, though it often converges more slowly than the original floating point version. Guidelines for a hardware implementation are provided, by analyzing how XCSi functional components can be designed on an FPGA.

Gradient Descent Methods in Learning Classifier Systems: Improving XCS Performance in Multistep Problems

The accuracy-based XCS classifier system has been shown to solve typical data mining problems in a machine-learning competitive way. However, successful applications in multistep problems, modeled by a Markov decision process, were restricted to very small problems. Until now, the temporal difference learning technique in XCS was based on deterministic updates. However, since a prediction is actually generated by a set of rules in XCS and Learning Classifier Systems in general, gradient-based update methods are applicable. The extension of XCS to gradient-based update methods results in a classifier system that is more robust and more parameter independent, solving large and difficult maze problems reliably. Additionally, the extension to gradient methods highlights the relation of XCS to other function approximation methods in reinforcement learning.