Theoretical Analysis of Accuracy-Based Fitness on Learning Classifier Systems

Abstract

Like most evolutionary algorithms, accuracy-based learning classifier systems (XCSs) use a fitness metric to recognize the superiority of rules, under a principle that a higher-quality rule has a higher fitness. However, XCS must learn the fitness values under a reinforcement learning scheme. This introduces uncertainty and asynchrony to the fitness estimation while no theoretical work formally guarantees that such a basic principle would hold. The goal of this paper is to complement this fundamental lack in the reliability of XCS by mathematically analyzing its fitness-update scheme. Our main assumption is that the fitness is updated with an absolute accuracy instead of its relative accuracy to sidestep unpredictable dynamics of XCS. Our theoretical conclusion is that the superiority of rules can be correctly recognized through the accuracy-based fitness under finite update times. We further show that recognizing the superiority among low-quality rules is a costly procedure that increases the number of necessary rule-trainings. This drawback indicates that XCS may struggle to identify good parent rules at early generations, degrading the efficiency of evolutionary propagation.