Hand Gesture Recognition Based on Trajectories Features and Computation-Efficient Reused LSTM Network

Abstract

Touchless hand gesture recognition using radar sensor is considered to be an attractive and effective technique for human-machine interaction. In this paper, we propose a hand gesture recognition approach based-on range-Doppler-angle trajectories and a reused long short-term memory (RLSTM) network using a 77GHz frequency modulated continuous wave (FMCW) multiple-input-multiple-output (MIMO) radar. To overcome amounts of gesture interferences, a gesture desktop is designed and a potential hand gesture detection approach is followed to determine whether it is a potential hand gesture. If the gesture happens in the assumed gesture desktop, the range-Doppler-angle trajectories are extracted by using the discretize Fourier transform (DFT), multiple signal classification (MUSIC) algorithm and the Kalman filtering. These signatures cannot only significantly reduce the dimension for the following neural network but also exploit the three dimensional information provided by the MIMO radar. Finally, a LSTM network with the reused forward propagation approach is proposed to exploit the spatial, temporal and Doppler information and simplify the network architecture. The experimental results show that the proposed approach can achieve an average accuracy of 99.4% and recognize a new person's hand gestures with an average accuracy of 98.0% for 9 hand gestures.