Internalizing Knowledge for Anticipatory Classifier Systems in Discretized Real-Valued Environments

Abstract

Real-valued environments are challenging for learning systems because of a significant increase in the input space size of the problem. This work demonstrates that Anticipatory Learning Classifier Systems (ALCS) can successfully build sets of conditional rules foreseeing the consequences of executed actions. Three major classes of Learning Classifier Systems - Anticipatory Classifier System (ACS), ACS2, Yet Another Classifier System (YACS), alongside the traditional Dyna-Q algorithm implementations were adapted to handle real-valued input signal discretization. Aspects like the ability to capture all possible interactions, model generalization capabilities, size of the solution of relative execution times were compared in four different problems using probabilistic modelling, providing unbiased judgments. Results proved that the examined ALCS are capable of solving selected problems. Despite increased input size, all possible environmental transitions were learned latently, without obtaining any explicit incentives. Such an internal representation provides a more compact solution representation and can optimize learning speed further by executing imaginary environmental interactions or performing action planning for a new set of potential problems.