Cross-Modal Discrete Hashing

Abstract

In this paper, we present a new cross-modal discrete hashing (CMDH) approach to learn compact binary codes for cross-modal multimedia search. Unlike most existing cross-modal hashing methods which usually relax the optimization objective function to obtain hash codes, we develop a discrete optimization framework to jointly learn binary codes and a series of hash functions for each modality, so that the performance drop due to the inferior optimization techniques can be avoided. Specifically, we present two cross-modal hashing algorithms called CMDH-linear and CMDH-kernel under the proposed framework, which performs linear and non-linear mappings to learn binary codes, respectively. Different from existing cross-modal hashing methods which maximize the corrections of hash codes from different modalities, our CMDH learns a set of shared binary codes for samples captured from different modalities, so that the modality gap can be effectively removed in cross-modal multimedia retrieval. To further improve the flexibility of our approach for different scenarios, we extend CMDH to unsupervised CMDH (unCMDH) and discrete multi-modal hashing (MMDH), which learns hash codes for training data without label information and with multi-modal labelled data. Experimental results on three benchmark datasets clearly show that our methods achieve competitive results with the state-of-the-arts.