PointNet+LSTM for Target List-Based Gesture Recognition With Incoherent Radar Networks

Abstract

Radar-based gesture recognition can provide autonomous electronic systems with a reliable way to infer a human's intention, e.g., in traffic environments involving vulnerable road users. Particularly in complex scenarios, algorithms operating on radar target lists derived from constant false-alarm rate outputs present an attractive solution, as they not only enable the filtering of relevant targets, but can also make full use of the diverse, high-resolution target parameters provided by modern radar sensors. Therefore, this article proposes PointNet+long short-term memory (LSTM) for the enhanced target list-based recognition of challenging traffic gestures, combining per-frame feature extraction with PointNet and learning from sequences with a LSTM. The approach is generalized to facilitate the use of multistatic radar data from sensor networks to exploit slightly different viewing angles, which is particularly helpful for motions with low radial velocity. The proposed method is validated on a comprehensive dataset comprising eight traffic gestures and data recorded from 35 participants. Measurements are conducted both indoors and outdoors with an incoherent radar sensor network comprising three chirp sequence–multiple-input multiple-output sensors. On this challenging dataset, our approach clearly outperforms a reference convolutional neural network, reaching up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$92.2 \%$</tex-math></inline-formula> cross-validation accuracy.