User-Definable Dynamic Hand Gesture Recognition Based on Doppler Radar and Few-Shot Learning

Abstract

In recent years, the radar-based dynamic hand gesture recognition (DHGR) system has been widely used in the non-contact interaction with smart electronic devices because of its advantages of safety, privacy security and robustness to different illumination environments. In order to achieve high-precision gesture recognition, Machine Learning algorithms, especially the deep learning algorithms are generally chosen by researchers. However, most deep learning models are trained based on large gesture dataset and only accept certain predefined specified gestures as input, i.e., user-defined gestures are not allowed. In this work, a neural network model trained with meta-learning method is proposed to deal with the few-shot classification task and realize the user-definable DHGR. Instead of learning a mapping between the unknown input gesture and the fixed predefined classes, the proposed model is trained to learn to compare the input gesture with arbitrary gestures entered by users, and then decide which class of gesture is the most probable one. Without retraining, the model can allow the user to enter the self-defined gestures for one or a few times and then recognize these gestures next time. To the best of our knowledge, this is the first work to apply the meta few-shot learning to the DHGR problem in the true sense. In this model, an embedding module is used in the network to extract the features of the input micro-Doppler spectrograms, and a comparison module is used to execute the feature-level comparison. Finally, a weighting module is proposed to weigh the comparison results and make the prediction. The evaluation result shows that with only one support sample for each class (i.e., for the one-shot tasks), the proposed model achieves an accuracy of 91% for 5 gesture classes and 90% for 10 gesture classes. When using 3 support samples (i.e., for the 3-shot tasks), the accuracies for 5 and 10 gestures are further improved to 96% and 92%, respectively.