Genetics-based Machine Learning System Research Articles

GAssist vs. BioHEL: critical assessment of two paradigms of genetics-based machine learning

This paper reports an exhaustive analysis performed over two specific Genetics-based Machine Learning systems: BioHEL and GAssist. These two systems share many mechanisms and operators, but at the same time, they apply two different learning paradigms (the Iterative Rule Learning approach and the Pittsburgh approach, respectively). The aim of this paper is to: (a) propose standard configurations for handling small and large datasets, (b) compare the two systems in terms of learning capabilities, complexity of the obtained solutions and learning time, (c) determine the areas of the problem space where each one of these two systems performs better, and (d) compare them with other well-known machine learning algorithms. The results show that it is possible to find standard configurations for both systems. With these configurations the systems perform up to the standards of other state-of-the-art machine learning algorithms such as Support Vector Machines. Moreover, we identify the problem domains where each one of these systems have advantages and disadvantages and propose ways to improve the systems based on this analysis.

Effective search for genetic-based machine learning systems via estimation of distribution algorithms and embedded feature reduction techniques

Genetic-based machine learning (GBML) systems, which employ evolutionary algorithms (EAs) as search mechanisms, evolve rule-based classification models to represent target concepts. Compared to Michigan-style GBML, Pittsburgh-style GBML is expected to achieve more compact solutions. It has been shown that standard recombination operators in EAs do not assure an effective evolutionary search to solve sophisticated problems that contain strong interactions between features. On the other hand, when dealing with real-world classification tasks, irrelevant features not only complicate the problem but also incur unnecessary matchings in GBML systems, which increase the computational cost a lot. To handle the two problems mentioned above in an integrated manner, a new Pittsburgh-style GBML system is proposed. In the proposed method, classifiers are generated and recombined at two levels. At the high level, classifiers are recombined by rule-wise uniform crossover operators since each classifier consists of a variable-size rule set. At the low level, single rules contained in classifiers are reproduced via sampling Bayesian networks that characterize the global statistical information extracted from promising rules found so far. Furthermore, according to the statistical information in the rule population, an embedded approach is presented to detect and remove redundant features incrementally following the evolution of rule population. Results of empirical evaluation show that the proposed method outperforms the original Pittsburgh-style GBML system in terms of classification accuracy while reducing the computational cost. Furthermore, the proposed method is also competitive to other non-evolutionary, highly used machine learning methods. With respect to the performance of feature reduction, the proposed embedded approach is able to deliver solutions with higher classification accuracy when removing the same number of features as other feature reduction techniques do.

Analysing BioHEL using challenging boolean functions

In this work we present an extensive empirical analysis of the BioHEL genetics-based machine learning system using the k-Disjunctive Normal Form (k-DNF) family of boolean functions. These functions present a broad set of possible challenges for most machine learning techniques, such as different degrees of specificity, class imbalance and niche overlap. Moreover, as the ideal solutions are known, it is possible to assess if a learning system is able to find them, and how fast. Specifically, we study two aspects of BioHEL: its sensitivity to the coverage breakpoint parameter (that determines the degree of generality pressure applied by the fitness function) and the impact of the default rule policy. The results show that BioHEL is highly sensitive to the choice of coverage breakpoint and that using a default class suitable for the problem allows the system to learn faster than using other default class policies (e.g. the majority class policy). Moreover, the experiments indicate that BioHEL’s scalability depends directly on both k (the specificity of the k-DNF terms) and the number of terms in the problem. In the last part of the paper we discuss alternative policies to adjust the coverage breakpoint parameter.

Application of Classifier System and Co-Evolutionary Algorithm in Optimization of Medium-Voltage Distribution Networks Post-Fault Configuration

The methodology of restoring power supply to consumers in case of a distribution grid failure is an important issue in literature on operation and reliability of electrical energy distribution grids. The article presents a concept of a new method which supports the operation of distribution grids operators in case of a failure using a classifier system working in tandem with a coevolutionary algorithm. The method presented in the article enables constructing scenarios for adjustments to medium voltage power distribution grids configuration (power system switchgear operation adjustments). The developed method is based on the theoretical rudiments of genetic-based machine learning systems. The method uses information on previous power distribution grid failures and enables using information from simulated grid failures. Decision variables, which take into account the reliability parameters of distribution grid elements among other things, have been described using the fuzzy set theory. The final part of the article describes sample calculations related to failures of a selected actual power distribution grid performed using the developed method. Ill. 5, bibl. 12, tabl. 1 (in English; abstracts in English and Lithuanian). http://dx.doi.org/10.5755/j01.eee.115.9.740

Genetic-based machine learning systems are competitive for pattern recognition

During the last decade, research on Genetic-Based Machine Learning has resulted in several proposals of supervised learning methodologies that use evolutionary algorithms to evolve rule-based classification models. Usually, these new GBML approaches are accompanied by little experimentation and there is a lack of comparisons among different proposals. Besides, the competitiveness of GBML systems with respect to non-evolutionary, highly-used machine learning techniques has only been partially studied. This paper reviews the state of the art in GBML, selects some of the best representatives of different families, and compares the accuracy and the interpretability of their models. The paper also analyzes the behavior of the GBML approaches with respect to some of the most influential machine learning techniques that belong to different learning paradigms such as decision trees, support vector machines, instance-based classifiers, and probabilistic classifiers. The experimental observations emphasize the suitability of GBML systems for performing classification tasks. Moreover, the analysis points out the strengths of the different systems, which can be used as recommendation guidelines on which systems should be employed depending on whether the user prefers to maximize the accuracy or the interpretability of the models.

Classifier systems in combat: two-sided learning of maneuvers for advanced fighter aircraft

This paper reports the continuing results of a project where a genetics-based machine learning system acquires rules for novel fighter combat maneuvers through simulation. In this project, a genetics-based machine learning system was implemented to generate high angle-of-attack air combat tactics for advanced fighter aircraft. This system, which was based on a learning classifier system approach, employed a digital simulation model of one-versus-one air combat, and a genetic algorithm, to develop effective tactics for the X-31 experimental fighter aircraft. Previous efforts with this system showed that the resulting maneuvers allowed the X-31 to successfully exploit its post-stall capabilities against a conventional fighter opponent. This demonstrated the ability of the genetic learning system to discover novel tactics in a dynamic air combat environment. The results gained favorable evaluation from fighter aircraft test pilots. However, these pilots noted that the static strategy employed by the X-31's opponent was a limitation. In response to these comments, this paper reports new results with two-sided learning, where both aircraft in a one-versus-one combat scenario use genetics-based machine learning to adapt their strategies. The experiments successfully demonstrate both aircraft developing objectively interesting strategies. However, the results also point out the complexity of evaluating results from mutually adaptive players, due to the red queen effect. These complexities, and future directions of the project, are discussed in the paper's conclusions.

A genetic-based machine learning system to discover the diagnostic rules for female urinary incontinence

A machine learning system named Galactica has been developed which uses a genetic algorithm to discover the rules for an expert system from databases. Galactica devised accurate diagnostic rules for female urinary incontinence from difficult heterogeneous data. The percentages of correctly classified stress, mixed and sensory urge incontinence testing cases were 89, 86 and 87%, respectively. However, these rules were rather general, consisting of 4–6 out of 13 conditions available in the data. Diagnostic rules for stress and mixed incontinence extracted from straightforward homogeneous data were highly accurate, classifying 100% of testing cases correctly as well as being specific, having from 10 to 11 conditions. More specific, but less accurate, rules were found from heterogeneous data with a biased fitness function. All of the rules were correct, i.e. every condition in the rules had the expected value specified by the expert. Although, Galactica achieved a slightly better classification than the discriminant analysis, it is argued that the genetic approach is better than the statistical one, due to symbolic rules being comprehensible, whereas understanding a complex mathematical model requires statistical expertise.

A Study on Evolutionary Synthesis of Classifier System Architectures.

This paper describes a general method to design architectures of reinforcement learning systems. The task of these systems is to create a stimulus-response pattern by which the expected longterm total reward is maximized. Reinforcement learning systems have high applicability to a broad task class of autonomous agents because of their flexibility and autonomy. However, it is difficult to determine the relevant set of learning parameters for a given task. These parameters dominate the system architecture and largely affect the learning performance. Therefore, we propose a new approach involving evolutionary synthesis of simple classifier system architectures, which is known as a genetics-based machine learning system. This synthesis mechanism is realized using genetic algorithms. To examine the validity of our proposed method, the evolutionary synthesis technique is applied to motion planning tasks of a robot manipulator. Results from computer simulation indicate the usefulness of this approach.

Emergent computation and the modeling and management of ecological systems

Abstract This paper introduces the emergent computational paradigm, discusses its applicability and potential in ecosystem management, and reviews the literature. Emergent computation is significantly different from the “classic” computational paradigm, where control is top-down and centralized. In emergent systems, overall system dynamics emerge from the local interactions of independent agents. In such systems, overall global control is minimized or eliminated altogether. Applications in ecosystem management include use of “artificial ecosystems” as surrogate experimental systems, and genetics-based machine learning systems to evolve management rule-sets for complex domains. Cellular automata, neural networks, genetic algorithms and classifier systems are discussed as examples of the emergent approach. Finally, an in-depth literature review of artificial ecosystems is provided.

Searching for Diverse, Cooperative Populations with Genetic Algorithms

In typical applications, genetic algorithms (GAs) process populations of potential problem solutions to evolve a single population member that specifies an ‘optimized’ solution. The majority of GA analysis has focused on these optimization applications. In other applications (notably learning classifier systems and certain connectionist learning systems), a GA searches for a population of cooperative structures that jointly perform a computational task. This paper presents an analysis of this type of GA problem. The analysis considers a simplified genetics-based machine learning system: a model of an immune system. In this model, a GA must discover a set of pattern-matching antibodies that effectively match a set of antigen patterns. Analysis shows how a GA can automatically evolve and sustain a diverse, cooperative population. The cooperation emerges as a natural part of the antigen-antibody matching procedure. This emergent effect is shown to be similar to fitness sharing, an explicit technique for multimodal GA optimization. Further analysis shows how the GA population can adapt to express various degrees of generalization. The results show how GAs can automatically and simultaneously discover effective groups of cooperative computational structures.

Using transputers to increase speed and flexibility of genetics-based machine learning systems

We implemented a distributed environment for machine learning experimentation on a transputer network. The system can be used by a researcher to build modular and efficient learning systems. The algorithms composing the basic structure of the implementation are the genetic algorithm, the bucket brigade algorithm and the inferential engine. We present a parallel version of these algorithms and call it low-level parallelism. Compared to the standard sequential version of the same algorithms, low-level parallelism gives us an increase in performance. To provide the learning system designer with a higher level of flexibility than currently available with standard systems, we also implemented high-level parallelism: subsets of the transputer network can be allocated to different learning systems. In this way a complex learning problem can be decomposed in many simpler problems, each one mapped on a single (possibly low-level parallel) learning system.