Human Interaction Recognition Based on Deep Learning and HMM

An Gong,Chen Chen,Mengtang Peng

doi:10.1109/access.2019.2951937

Abstract

This paper proposes a recognition method that combines deep learning with traditional hidden Markov model (HMM) with the aim of improving the recognition accuracy of interaction. First, to construct the classification model, the optimized ALexNet convolutional neural network is used to extract the behavior features, followed by the extraction of features that are used to train the long short-term memory (LSTM) network using the Softmax method. Finally, the particle swarm optimization algorithm is used to fuse the classification results with the traditional HMM classification results so that a hybrid classification model is established to obtain the final behavior recognition result. By conducting experiments on the UT-interaction dataset (six types of interaction behavior), the experimental results show that the hybrid model has higher recognition accuracy than other classical methods.

Highlights

Behavior recognition has increasing research attention due to its wide application potential in related fields, such as video surveillance, human–computer interaction, and social video recommendation
This paper presents a method based on the combination of traditional methods and deep learning, and proposes a behavior recognition model based on the combination of deep learning and hidden Markov model (HMM)
The convolutional neural networks (CNNs) structure adopts the ALexNet network after optimization, the long short-term memory (LSTM) adopts a single-layer LSTM, and Softmax is used for the classification

Summary

INTRODUCTION

Behavior recognition has increasing research attention due to its wide application potential in related fields, such as video surveillance, human–computer interaction, and social video recommendation. This paper presents a method based on the combination of traditional methods and deep learning, and proposes a behavior recognition model based on the combination of deep learning and hidden Markov model (HMM) In this model, the CNN structure adopts the ALexNet network after optimization, the LSTM adopts a single-layer LSTM, and Softmax is used for the classification. Eight, the effect is not necessarily better than the five key frames, and the training time is increased It affects the accuracy of behavioral expression and the efficiency of processing data, so a total of 30 key frame images are selected for the six types of behavior in the UT-interaction data set. HIDDEN MARKOV CLASSIFIER HMM is an effective method for modeling small changes in space–time motion based on probabilistic methods It is widely used in the study of interactive behavior recognition. The image whose behavior is not obvious in each behavior category is deleted, and the remaining image collection is used for the neural network training

ALexNet CONVOLUTIONAL NEURAL NETWORK

LONG SHORT-TERM MEMORY

VERIFICATION OF THE DEEP NETWORK EXPERIMENT PERFORMANCE

Findings

CONCLUSION