DeepSegment: Segmentation of motion capture data using deep convolutional neural network

Abstract

In this paper, we propose a novel framework to segment 3D human motion capture data into distinct behaviors. First, in preprocessing, we build a normalized pose space by eliminating translation and orientation from the 3D poses. We then transform these normalized 3D poses into 2D RGB images, and as a result, we simplify the task of motion segmentation as image classification and recognition. Furthermore, we identify the most significant joints of the skeleton that contribute substantially to executing a motion and get benefits from them by assigning them more weights. The weight allocation to the specific joint has been done purely based on its deviation capability. Finally, each motion is encoded into compact visual representation by exploiting RGB images with weighted joints. We adopt a transfer learning approach to extract a fixed-size feature vector using off-the-shelf deep Convolutional Neural Network (CNN), Alexnet, after fine-tuning. We develop a Kd-tree on these highly descriptive feature vectors to retrieve the nearest neighbors. Based on a similarity measure, we classify the motion segments and ultimately place the cuts on the ongoing motion sequences. We perform extensive experiments to evaluate our proposed approach on popular Motion Capture (MoCap) datasets, CMU and HDM05. Our approach almost outperforms all other state-of-the-art methods, and the results highlight the capabilities of our proposed scheme for effective segmentation.