Convolutional Neural Networks Based Motion Data Optimization Networks forLeap Motion

Abstract

<p indent=0mm>It is necessary to improve the capture accuracy and precision of Leap Motion equipment, and solve its inherent problems such as finger self-occlusion and instable sampling frequency, firstly, a multi-modal synchronous hand motion capture scheme is proposed based on Leap Motion and motion capture devices, and the dataset is captured correspondingly. Secondly, a convolutional-neural-network-based hand motion data optimization method for Leap Motion is presented. The proposed network is trained to learn the mapping from Leap Motion data domain to motion capture data domain with the synchronous dataset. Finally, a coplanar constraint for human fingers is proposed, which makes the outputs maintain stable hand skeleton structure. The 967 550 frames of synchronous hand motion data are captured in 6 categories from 15 volunteers. Experiments are designed for validating of the finger coplanar constraint, testing the consistency of the optimized motion data, and the comparison with BRA (bidirectional recurrent autoencoder) and EBD (encoder-bidirectional-decoder) methods. Experiments indicate that proposed method supports for capturing hand motion data with fixed sample frequency and MoCap-like precision using a Leap Motion. Furthermore, the proposed method can obtain smoother and more stable results than BRA and EBD. The method is also applied in rehabilitation games, which significantly reduces the number of errors of hand interaction motion recognition.