An integrated architecture for learning of reactive behaviors based on dynamic cell structures

Abstract

In this contribution we want to draw the readers attention to the advantages of dynamic cell structures (DCSs) (Bruske and Sommer, 1995) for learning reactive behaviors of autonomous robots. These include incremental on-like learning, fast output calculation, a flexible integration of different learning rules and a close connection to fuzzy logic. The latter allows for incorporation of prior knowledge and to interpret learning with DCSs as fuzzy rule generation and adaptation. After successful applications of DCSs to tasks involving supervised learning, feedback error learning and incremental category learning, in this article we take reinforcement learning of reactive collision avoidance for an autonomous mobile robot as a further example to demonstrate the validity of our approach. More specifically, we employ a REINFORCE (Williams, 1992) algorithm in combination with an adaptive heuristic critique (AHC) (Sutton, 1988) to learn a continuous valued sensory motor mapping for obstacle avoidance with a TRC Labmate from delayed reinforcement. The sensory input consists of eight unprocessed sonar readings, the controller output is the continuous angular and forward velocity of the Labmate. The controller and the AHC are integrated within a single DCS network, and the resulting avoidance behavior of the robot can be analyzed as a set of fuzzy rules, each rule having an additional certainty value.