Learning spatial-temporal features for video copy detection by the combination of CNN and RNN

Abstract

Listen

Translate article

Following the rapid developments of network multimedia, video copyright protection online has become a hot topic in recent researches. However, video copy detection is still a challenging task in the domain of video analysis and computer vision, due to the large variations in scale and illumination of the copied contents. In this paper, we propose a novel deep learning based approach, in which we jointly use the Convolution Neural Network (CNN) and Recurrent Neural Network (RNN) to solve the specific problem of detecting copied segments in videos. We first utilize a Residual Convolutional Neural Network(ResNet) to extract content features of frame-levels, and then employ a SiameseLSTM architecture for spatial-temporal fusion and sequence matching. Finally, the copied segments are detected by a graph based temporal network. We evaluate the performance of the proposed CNN-RNN based approach on a public large scale video copy dataset called VCDB, and the experiment results demonstrate the effectiveness and high robustness of our method which achieves the significant performance improvements compared to the state of the art.