Unsupervised Variational Video Hashing With 1D-CNN-LSTM Networks

Abstract

Most existing unsupervised video hashing methods generate binary codes by using RNNs in a deterministic manner, which fails to capture the dominant latent variation of videos. In addition, RNN-based video hashing methods suffer the content forgetting of early input frames due to the sequential processing inherency of RNNs, which is detrimental to global information capturing. In this work, we propose an unsupervised variational video hashing (UVVH) method for scalable video retrieval. Our UVVH method aims to capture the salient and global information in a video. Specifically, we introduce a variational autoencoder to learn a probabilistic latent representation of the salient factors of video variations. To better exploit the global information of videos, we design a 1D-CNN-LSTM model. The 1D-CNN-LSTM model processes long frame sequences in a parallel and hierarchical way, and exploits the correlations between frames to reconstruct the frame-level features. As a consequence, the learned hash functions can produce reliable binary codes for video retrieval. We conduct extensive experiments on three widely used benchmark datasets, FCVID, ActivityNet and YFCC to validate the effectiveness of our proposed approach.