Granular Computing in Actor-Critic Learning

Abstract

The problem considered in this paper is how to guide actor-critic learning based an information granules that reflect knowledge about acceptable behavior patterns. The solution to this problem stems from approximation spaces, which were introduced by Zdzistaw Pawlak starting in the early 1980s and which provide a basis for perception of objects that are imperfectly known. It was also observed by Ewa Orlowska En 1982 that approximation spaces serve as a formal counterpart of perception, or observation. In our case, approximation spaces provide a ground for deriving pattern-based behaviours as well as information granules that can be used to influence the policy structure of an actor in a beneficial way. This paper includes the results of a recent study of swarm behavior by collections of biologically-inspired bots carried out in the context of an artificial ecosystem. This ecosystem has an ethological basis that makes it possible to observe and explain the behavior of biological organisms that carries over into the study of actor-critic learning by interacting robotic devices. The contribution of this article is a framework for actor-critic learning defined in the context of approximation spaces and information granulation