One-shot learning based pattern transition map for action early recognition

Abstract

In a natural and accessible Human Robot Interaction (HRI), it is required to understand human activities instantly. In this paper, we present a novel approach for early recognition of human actions. Using reinforcement learning, we separate human action to several patterns and learn pattern transition maps which include temporal ordered patterns and their transition relationships in action sequences. Due to the difficulty of pattern separation and definition in large quantity of action sequences for training, we adopt one-shot learning to automatically define patterns. Moreover, we propose a pattern transition map based soft-regression approach for early recognition. We evaluate the proposed approach on the MSR Action Pairs Database and the SYSU 3DHOI database. Experiments show that our approach recognizes ongoing sequences with high accuracies. Compared with state-of-the-art approaches, our proposed approach also obtain encouraging results for full action sequence recognition.