Human Action Recognition by Fusion of Convolutional Neural Networks and spatial-temporal Information

Abstract

In order to improve the accuracy of recognizing human action, a human action recognition model is proposed based on improved convolutional neural networks. Most of the current methods build classifiers based on complex handcrafted features computed from the raw inputs. Convolutional neural network (CNN) is a type of deep model that can act directly on the original inputs. In this paper we modify the existing network structure for action recognition, and then develop a different 3D CNN models in order to fuse the information of spatial and temporal dimensions. What's more we propose the synthesis silhouette images using classic 2D CNN, which choose a suitable frames image instead of the whole movement process. Majority voting is used to produce labels for a video sequence based on the predictions for individual frames. We evaluate our method on the DHA and KTH datasets. Experiment shows the effectiveness of the proposed method.