Action recognition using micro-Doppler signatures and a recurrent neural network

Abstract

This paper explores the long short-term memory (LSTM) recurrent neural network for human action recognition from micro-Doppler signatures. The recurrent neural network model is evaluated using the Johns Hopkins MultiModal Action (JHUMMA) dataset. In testing we use only the active acoustic micro-Doppler signatures. We compare classification performed using hidden Markov model (HMM) systems trained on both micro-Doppler sensor and Kinect data with LSTM classification trained only on the micro-Doppler signatures. For HMM systems we evaluate the performance of product of expert based systems and systems trained on concatenated sensor data. By testing with leave one user out (LOUO) cross-validation we verify the ability of these systems to generalize to new users. We find that LSTM systems trained only on micro-Doppler signatures outperform the other models evaluated.