Reinforcement Learning in Swarms that Learn

Abstract

This paper introduces an approach to reinforcement learning by cooperating agents using a variation of the actor critic method. This is made possible by considering behavior patterns of swarms in the context of approximation spaces. Rough set theory introduced by Zdzislaw Pawlak in 1982 provides a ground for deriving pattern-based rewards within approximation spaces. The framework provided by an approximation space makes it possible to derive pattern-based reference rewards used to estimate action preferences. Approximation spaces are used to derive action-based reference rewards at the swarm intelligence level. Two different forms of the actor critic reinforcement learning method are considered as a part of a study of learning in real-time by a swarm. The contribution of this article is the presentation of a new actor critic method defined in the context of approximation spaces. An ecosystem designed to facilitate study of reinforcement learning by swarms is briefly described. In addition, the results of ecosystem experiments for two forums of the actor critic method are given.