Action Recognition Based on Linear Dynamical Systems with Deep Features in Videos

Abstract

We propose a new framework that boasts of low training costs and high generalization performance in capturing human action expressions simultaneously on spatial and temporal structures. First, a video slicing process is established. Then, in order to capture the divergence and likelihood expression of spatial structure in each video slice, a pipeline is introduced using a pre-trained CNN. In addition, any pre-trained network can be used to extract these features. Subsequently, Linear Dynamical Systems (LDS) is established to determine the timing relationship between two adjacent slices to obtain the temporal structure of divergence and likelihood features, which are expressed as LD-Divergence and LD-Likelihood. In UCF50 and UCF101 datasets, we analyzed the impact of different feature dimensions retained by PCA on recognition. Finally, we combined LD-Divergence and LD-Likelihood to improve accuracy to 0.961 and 0.949 on UCF50 and UCF101 datasets. Experimental results show that proposed framework simultaneously expresses spatial and temporal structures, which in turn produce state-of-the-art results.