Human Activity Classification Based on Micro-Doppler Signatures by Multiscale and Multitask Fourier Convolutional Neural Network

Abstract

Human detection and activity classification has recently become a key technology in many applications, e.g., human computer interaction and surveillance for public and industrial security. In this work, we propose a novel end-to-end deep learning-based framework called the Fourier convolutional neural network (F-Convents) to tackle this problem. The input of F-ConvNet consists of raw frames of radar data. It is fed to a new layer called the Fourier layer, which transforms the raw radar signal into a domain optimized for the classification task. A novel weight initialization method tailored for the Fourier layer is also proposed. Moreover, we use dilated convolutions to further improve both performance and efficiency. To achieve better convergence and accuracy, a multi-scale and multi-task loss consisting of cross-entropy and triplet loss with a novel training paradigm called dynamic training is proposed. Experimental results show that F-ConvNet surpasses state-of-the-art methods by 3% in terms of classification accuracy.