Deep semantic cross modal hashing with correlation alignment

Abstract

Hashing has been extensively applied to cross modal retrieval due to its low storage and high efficiency. Deep hashing which can well extract features of multi-modal data has received increasing research attention recently. However, most of deep hashing for cross modal retrieval methods do not make full use of the semantic label information and do not fully mine correlation of heterogeneous data. In this paper, we propose a Deep Semantic cross modal hashing with Correlation Alignment (DSCA) method. In DSCA, we design two deep neural networks for image and text modality separately, and learn two hash functions. Firstly, we construct a new similarity for the multi-label data, which can well exploit the semantic information and improve the retrieval accuracy. Simultaneously, we preserve the inter-modal similarity of heterogeneous data features, which can exploit semantic correlation. Secondly, the distributions of heterogeneous data are aligned so as to mine the inter-modal correlation well. Thirdly, the semantic label information is embedded in the hash layer of the text network, which can make the learned hash matrix more stable and make the hash codes more discriminative. Experimental results demonstrate that DSCA outperforms the state-of-the-art methods.