Understanding human intention by connecting perception and action learning in artificial agents

Abstract

To develop an advanced human–robot interaction system, it is important to first understand how human beings learn to perceive, think, and act in an ever-changing world. In this paper, we propose an intention understanding system that uses an Object Augmented-Supervised Multiple Timescale Recurrent Neural Network (OA-SMTRNN) and demonstrate the effects of perception–action connected learning in an artificial agent, which is inspired by psychological and neurological phenomena in humans. We believe that action and perception are not isolated processes in human mental development, and argue that these psychological and neurological interactions can be replicated in a human–machine scenario. The proposed OA-SMTRNN consists of perception and action modules and their connection, which are constructed of supervised multiple timescale recurrent neural networks and the deep auto-encoder, respectively, and connects their perception and action for understanding human intention. Our experimental results show the effects of perception–action connected learning, and demonstrate that robots can understand human intention with OA-SMTRNN through perception–action connected learning.