Human Motion Recognition With Spatial-Temporal-ConvLSTM Network Using Dynamic Range-Doppler Frames Based on Portable FMCW Radar

Abstract

As a noncontact and noninvasive sensor, radar offers attractive modality for human motion recognition in many fields, such as surveillance, smart homes, and assisted living. As a dynamic process, a comprehensive multidomain analysis in stream is crucial for human motion recognition. In this article, a novel spatial–temporal convolution long short-term memory (ST-ConvLSTM) network using dynamic range-Doppler frames (DRDF) based on portable frequency-modulated continuous-wave (FMCW) radar is proposed to address this problem. First, DRDF is introduced to characterize different human motions in time, range, Doppler, and radar cross section (RCS) domains with a time-sequenced range-Doppler maps. Then, ConvLSTM cell is investigated to extract continuous 2-D image features by combining both convolution operation and LSTM cell. Furthermore, a spatial attention module is utilized to emphasize the spatial distribution of human motion in the range-Doppler domain, while a temporal attention module is designed to obtain optimal temporal weights considering behavior continuity. Finally, an extensive radar dataset from 16 volunteers demonstrated its feasibility and superiority in the classification of six typical daily human motions. The contributions of attention modules and its robustness facing individual diversity are also investigated and discussed.