Unsupervised Deep Cross-modal Hashing with Virtual Label Regression

Abstract

Unsupervised cross-modal hashing learns hash codes without dependence on the semantic labels. It has the desirable advantage of well scalability and thus can effectively support large-scale cross-media retrieval. However, how to directly learn discriminative discrete hash codes under the unsupervised learning paradigm is still an open challenging problem. In this paper, we aim to attack this problem by proposing an Unsupervised Deep Cross-modal Hashing with Virtual Label Regression (UDCH-VLR) method. We propose a novel unified learning framework to jointly perform deep hash function training, virtual label learning and regression. Specifically, we learn unified hash codes via collaborative matrix factorization on the multi-modal deep representations to preserve the multi-modal shared semantics. Further, we incorporate the virtual label learning into the objective functions and simultaneously regress the learned virtual labels to the hash codes. Finally, instead of simply exploiting the existing shallow features and relaxing the binary constraints, we devise an alternative optimization strategy to directly update the deep hash functions and discrete binary codes. Under such circumstance, the discriminative capability of hash codes can be progressively enhanced with iterative learning. Extensive experiments on three publicly available cross-media retrieval datasets demonstrate that our approach outperforms state-of-the-art methods.