A new spatial-temporal histograms of gradients descriptor and HOD-VLAD encoding for human action recognition

Abstract

Automatic human action recognition is a core functionality of systems for video surveillance and human object interaction. In the whole recognition system, feature description and encoding represent two crucial key steps. In order to construct a powerful action recognition framework, it is important that the two steps must provide reliable performance. In this paper, we proposed a new human action feature descriptor which is called spatio-temporal histograms of gradients (SPHOG). SPHOG is based on the spatial and temporal derivation signal, which extracts the gradient changes between consecutive frames. Compared to the traditional descriptors histograms of optical flow, our proposed SPHOG costs less computation resource. In order to incorporate the distribution information of local descriptors into Vector of Locally Aggregated Descriptors (VLAD), which is a popular encoding approach for Bag-of-Feature representation, a Gaussian kernel is implanted to compute the weighted distance histograms of local descriptors. By doing this, the encoding schema for bag-of-feature (BOF) representation is more effective. We validated our proposed algorithm for human action recognition on three public available datasets KTH, UCF Sports and HMDB51. The evaluation experiment results indicate that the proposed descriptor and encoding method can improve the efficiency of human action recognition and the recognition accuracy.