Rammar: RAM Assisted Mask R-CNN for FMCW Sensor Based HGD System

Abstract

Recently, hand gesture detection (HGD) system have become increasingly interesting to researchers in the field of human-computer interfaces. However, the traditional HGD has low robustness and detection accuracy, as well as privacy protection problem. Therefore, we present Rammar, a residual attention module (RAM) assisted Mask R-CNN, for frequency modulated continuous wave (FMCW) sensor based on HGD system. Firstly, by analyzing the time domain and frequency domain of the FMCW sensor signal, the three-dimensional feature maps of Range-Time-Map (RTM), Doppler-Time-Map (DTM) and Angle-Time-Map (ATM) of each hand gesture are obtained, respectively, avoiding insufficient information of single dimension parameter. Secondly, RTM, DTM and ATM images of each hand gesture are simultaneously sent to Rammar for training. To focus on the features of gesture, RAM in Rammar employs average-pooling and max-pooling to extract time and spatial features. Finally, the extracted three-dimensional feature maps are merged in the fully connected layer. The experimental results show that Rammar not only makes the average detection accuracy of the hand gestures to 98.1%(increased by 5%), but also reduces the detection time effectively.