Motion Primitives Classification Using Deep Learning Models for Serious Game Platforms

Abstract

Serious games are receiving increasing attention in the field of Cultural Heritage (CH) applications. A special field of CH and education is Intangible Cultural Heritage and particularly dance. Machine learning (ML) tools are necessary elements for the success of a serious game platform since they introduce intelligence in processing and analysis of users' interactivity. ML provides intelligent scoring and monitoring capabilities of the user's progress in a serious game platform. In this paper, we introduce a deep learning model for motion primitive classification. The model combines a convolutional processing layer with a bi-directional analysis module. This way, RGB information is efficiently handled by the hierarchies of convolutions, while the bi-directional properties of an LSTM model are retained. The resulting Convolutionally Enhanced Bi-directional LSTM (CEBi-LSTM) architecture is less sensitive to skeleton errors, occurring using low-cost sensors, such as Kinect, while simultaneously handling the high amount of detail when using RGB visual information.