Emulation and behavior understanding through shared values

Abstract

Neurophysiology has revealed the existence of mirror neurons in the brain of macaque monkeys that activate when the monkey executes a goal directed behavior and also when it observes the same behavior performed by another. The concept of the mirror neurons/systems (Oztop and Kawato, 2006) [19] suggests that behavior acquisition and understanding of others’ behavior are related. We propose a method not only to learn and execute a variety of behaviors but also to understand observed behavior, supposing that the observer has already acquired the utilities (state values in reinforcement learning scheme) of all kinds of behavior the observed agent can do. The method does not need a precise world model or a coordinate transformation system to deal with a view difference caused by different viewpoints. This paper shows that an observer can understand/recognize behaviors shown by a demonstrator based not on a precise object trajectory in allocentric/egocentric coordinate space but rather on an estimated utility transition during the observed behavior. Furthermore, it is shown that the loop of the behavior acquisition and recognition of observed behavior accelerates the learning and improves the recognition performance. The state value updates can be accelerated by the observation without real trial and error while the learned values enrich the recognition system, since it is based on estimation of the state value of the observed behavior. The validity of the proposed method is shown by applying it to a dynamic environment where two robots play soccer.