Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

Abstract

Deep learning makes radar-based human activity recognition (HAR) attract more attention in the fields of intelligent security, traffic management, medical rehabilitation and military operation, because it has the ability to automatically extract comprehensive features of human activity. This paper proposes a recognition method based on multi-spectrogram and mixed convolutional neural network (MCNN). Specifically, three time-frequency analyses, including short-time Fourier transform (STFT), reduced interference distribution with Hanning kernel (RIDHK) and smoothed pseudo Wigner-Ville distribution (SPWVD), are performed on radar echo data to obtain the time-frequency spectrograms with different feature expressions, and then the spectrograms are fed into the MCNN for recognition and classification. In the MCNN, three two-dimensional CNNs (2DCNNs) are used to extract the independent spatial features from three types of spectrograms, and one three-dimensional CNN (3DCNN) with unit convolution kernel is employed to focus on extracting the correlation features between three kinds of spectrograms. The behavioral features in the spectrograms are characterized comprehensively by fusing these two kinds of features, which is able to improve the recognition accuracy. Experimental results illustrate that the proposed method improves the average recognition accuracy of eight human activities by at least 2.16% compared with other methods.