Human Motion Recognition by Three-view Kinect Sensors in Virtual Basketball Training

Abstract

In recent years, human action recognition has received a considerable amount of research attention because of it's potential in a variety of applications, such as video surveillance, human-computer interaction, and virtual reality (VR). However, many researches on human action recognition performed in single-camera or double-camera system, which achieve reduced performance due to vulnerability to partial occlusion and miss-recognition of back. Some works on human action recognition use multiple cameras but are too complex for practical application. In this paper, we propose a new human action recognition system using triple Kinect sensors for VR application. Particularly, we design a mark detection method to determine the front of user and fusion skeleton data in real time. Features are extracted from three-dimensional (3D) skeleton data sequences, and divided into five parts according to body parts. A classification model based on the part-aware long short-term memory networks is proposed to recognize human motion. Finally, we demonstrate the system with a virtual reality basketball application and the results of experiment validate the feasibility of the proposed system.