Human Activity Recognition Based on Deep Learning with Multi-spectrogram

Abstract

This paper mainly deals with the problem of human activity recognition, by utilizing deep learning to exploit multi-resolution characteristics from multiple spectrograms. Specifically, three classical time-frequency analysis methods, namely short-time Fourier transform (STFT), reduced interference distribution with Hanning kernel (RIDHK) and smoothed pseudo Wigner-Ville distribution (SPWVD) are performed to generate three kinds of single spectrograms with different time-frequency resolutions. Then three single spectrograms are combined into a composite spectrogram which is fed into a typical deep convolutional neural network (DCNN), named VGG16 for activity recognition. Because of the more comprehensive representation for human activity, the composite spectrogram contributes to higher recognition accuracy in comparison to the single spectrogram. Experimental results demonstrate that the application of the composite spectrogram improves the averaging recognition accuracy of six activities by at least 1.5% compared to only using each of three single spectrograms.