Zeroth-level Classifier System Research Articles

Improving selection strategies in zeroth-level classifier systems based on average reward reinforcement learning

As a genetics-based machine learning technique, zeroth-level classifier system (ZCS) based on average reward reinforcement learning (ZCSAR) evolves solutions to optimize average reward per time step. However, initial experimental results have shown that, in some cases, the performance of ZCSAR oscillates heavily during the learning period, or cannot reach the optimum during the testing period. In this paper, we modify the selection strategies in ZCSAR to improve its performance, under conditions of minimal changes of ZCSAR. The proposed selection strategies take tournament selection method to choose parents in Genetic Algorithm (GA), and take roulette wheel selection method to choose actions in match set and to choose classifiers for deletion in both GA and covering. Experimental results show that ZCSAR with the new selection strategies can evolve more promising solutions with enough parameter independence, and also with slighter oscillation during the learning period.

Tournament Selection in Zeroth-Level Classifier Systems Based On Average Reward Reinforcement Learning

As a genetics-based machine learning technique, zeroth-level classifier system (ZCS) is based on a discounted reward reinforcement learning algorithm, bucket-brigade algorithm, which optimizes the discounted total reward received by an agent but is not suitable for all multi-step problems, especially large-size ones. There are some undiscounted reinforcement learning methods available, such as R-learning, which optimize the average reward per time step. In this paper, R-learning is used as the reinforcement learning employed by ZCS, to replace its discounted reward reinforcement learning approach, and tournament selection is used to replace roulette wheel selection in ZCS. The modification results in classifier systems that can support long action chains, and thus is able to solve large multi-step problems.

Analyzing Strength-Based Classifier System from Reinforcement Learning Perspective

Learning Classifier Systems (LCSs) are rule-based adaptive systems that have both Reinforcement Learning (RL) and rule-discovery mechanisms for effective and practical on-line learning. With the aim of establishing a common theoretical basis between LCSs and RL algorithms to share each field's findings, a detailed analysis was performed to compare the learning processes of these two approaches. Based on our previous work on deriving an equivalence between the Zeroth-level Classifier System (ZCS) and Q-learning with Function Approximation (FA), this paper extends the analysis to the influence of actually applying the conditions for this equivalence. Comparative experiments have revealed interesting implications: (1) ZCS's original parameter, the deduction rate, plays a role in stabilizing the action selection, but (2) from the Reinforcement Learning perspective, such a process inhibits the ability to accurately estimate values for the entire state-action space, thus limiting the performance of ZCS in problems requiring accurate value estimation.

분류자 시스템을 이용한 축구 로봇의 행동 전략

Learning Classifier System (LCS) finds a new rule set using genetic algorithm (GA). In this paper, The Zeroth Level Classifier System (ZCS) is applied to evolving the strategy of a robot soccer simulation game (SimuroSot), which is a state varying dynamical system changed over time, as GBML (Genetic Based Machine Learning) and we show the effectiveness of the proposed scheme through the simulation of robot soccer.

An accuracy based corporate classifier system

Previously a corporate classifier system has been implemented based on the ideas of Wilson and Goldberg which demonstrates that rule-linkage can, for a certain class of problems, offer benefits to a system based on the zeroth-level classifier system (ZCS). In this work it is shown that similar benefits can be gained when similar rule-linkage mechanisms are applied to XCS. The intention is that the resultant system (CXCS) will exhibit XCS's generalization capabilities along with CCS's abilities to solve non-Markov tasks.

Self-adaptation in classifier system controllers

The use and benefits of self-adaptive mutation operators are well known within evolutionary computing. In this paper, we begin by examining the use of self-adaptive mutation in learning classifier systems with the aim of improving their performance as controllers for autonomous mobile robots. We implement the operator in a zeroth level classifier system, and examine its performance in two animat environments. It is shown that although no significant increase in performance is seen over results presented in the literature using a fixed rate of mutation, the operator adapts to an appropriate rateregadless of the initial range. The same concept is then applied to the learning rate parameter, but results show that a modification must be made to produce stable/effective controllers. Finally, results from a fully self-adaptive system are presented, with marked benefits being found in a nonstationary environment.

Adding Temporary Memory to ZCS

In a recent article, Wilson (1994) described a "zeroth-level" classifier system (ZCS). ZCS employs a reinforcement learning technique comparable to Q-learning (Watkins, 1989). This article presents results from the first reconstruction of ZCS. Having replicated Wilson's results, we extend ZCS in a manner suggested by Wilson: The original formulation of ZCS has no memory mechanisms, but Wilson (1994b) suggested how internal "temporary memory" registers could be added. We show results from adding one-bit and two-bit memory registers to ZCS. Our results demonstrate that ZCS can exploit memory facilities efficiently in non-Markov environments. We also show that the memoryless ZCS can converge on near-optimal stochastic solutions in non-Markov environments. We then present results from trials using ZCS in Markov environments that require increasingly long chains of actions before reward is received. Our results indicate that inaccurate overgeneral classifiers can interact with the classifier-generation mechanisms to cause catastrophic breakdowns in overall system performance. Basing classifier fitness on accuracy may alleviate this problem. We conclude that the memory mechanism in its current form is unlikely to scale well for situations requiring large amounts of temporary memory. Nevertheless, the ability to find stochastic solutions when there is insufficient memory might offset this problem somewhat.