Dynamic Hand Gesture Recognition in In-Vehicle Environment Based on FMCW Radar and Transformer.

Lianqing Zheng,Libo Huang,Jie Bai,Qiong Wu,Chewu Shan,Xichan Zhu,Lei Zhang

doi:10.3390/s21196368

Abstract

Hand gesture recognition technology plays an important role in human-computer interaction and in-vehicle entertainment. Under in-vehicle conditions, it is a great challenge to design gesture recognition systems due to variable driving conditions, complex backgrounds, and diversified gestures. In this paper, we propose a gesture recognition system based on frequency-modulated continuous-wave (FMCW) radar and transformer for an in-vehicle environment. Firstly, the original range-Doppler maps (RDMs), range-azimuth maps (RAMs), and range-elevation maps (REMs) of the time sequence of each gesture are obtained by radar signal processing. Then we preprocess the obtained data frames by region of interest (ROI) extraction, vibration removal algorithm, background removal algorithm, and standardization. We propose a transformer-based radar gesture recognition network named RGTNet. It fully extracts and fuses the spatial-temporal information of radar feature maps to complete the classification of various gestures. The experimental results show that our method can better complete the eight gesture classification tasks in the in-vehicle environment. The recognition accuracy is 97.56%.

Highlights

IntroductionGesture recognition is one of the most important branches in the field of humancomputer interaction [2]
This paper considers gesture recognition in the in-vehicle environment, which belongs to the field of in-vehicle entertainment and intelligent cockpit
The input of the network consists of three branches, i.e., range-Doppler maps (RDMs), range-azimuth maps (RAMs) and range-elevation maps (REMs)

Summary

Introduction

Gesture recognition is one of the most important branches in the field of humancomputer interaction [2]. It has been widely used in industrial production [3,4], including smart homes, virtual reality, and intelligent cockpits. To remove the cluttered background gesture features, we use the we use Tointerference remove the of interference of cluttered on background on gesture features, frame-difference method of dynamic to remove the. We u unidirectional queue to save the sixteen background frames nearest the current time. Set unidirectional queue to save the sixteen background frames nearest the current time each background as B1 , B.

Methods

Results

Discussion

Conclusion