A novel F-RCNN based hand gesture detection approach for FMCW systems

Abstract

With the rapid development of artificial intelligence, hand gesture detection has gradually become a research hotspot in human–computer interaction. However, the traditional hand gesture detection has low robustness and detection accuracy, and brings the problem of privacy protection. Therefore, this paper proposes a Faster Region-based Convolutional Neural Network (F-RCNN) hand gesture detection method based on Frequency Modulated Continuous Wave (FMCW) radar with 3-Dimensions deep convolutional Generative Adversarial Networks (3-DCGAN). In particular, the range Doppler and angle of the hand gestures are calculated by FMCW radar. Then, the semantic label maps of the Range-Time-Map, Doppler-Time-Map and Angle-Time-Map images are respectively sent to the 3-DCGAN to expand the datasets. After that, the original and the 3-DCGAN generated images are sent to F-RCNN for jointly training. The classifier function is designed, and a learning ranking based assessment called Rank based Quality Score (RQS) is applied to improve the detection performance. The experimental results show that the mean average precision reaches to 80.8%. Moreover, the RQS for hand gesture detection is as high as 96.3%, which is increased by 5.8% compared to the traditional F-RCNN.