Deep Cross-view Convolutional Features for View-invariant Action Recognition

Abstract

Convolutional neural network (CNN) based approaches have proved very effective for recognizing actions from a fixed viewpoint. However, these approaches are not generalized for recognizing actions captured from arbitrary viewpoint. In this paper, we present a deep multi-view framework for cross-view action recognition. We integrate spatiotemporal convolutional features from multiple views using deep multi-view representation learning. It helps to extract deep discriminative cross-view convolutional features for action recognition from any arbitrary viewpoint. To speed-up action detection and recognition, we then, train a feature based correlation filter for each action class. The proposed framework helps to recognize actions across different view-points with increased accuracy. An extensive experimentation to evaluate the underlying design on four publicly available datasets indicates that problem of view variations in in a single action class can be solved by learning discriminative information from multiple view.