Adversarial Projection Learning Based Hashing for Cross-Modal Retrieval

Abstract

<p indent=0mm>Cross-modal Hashing has received a lot of attentions in the field of cross-modal retrieval due to its high retrieval efficiency and low storage cost. Most of the existing cross-modal Hashing methods learn Hash codes directly from multimodal data and cannot fully utilize the semantic information of the data, so the distribution consistency of low-dimensional features across modalities cannot be guaranteed. To this end, adversarial projection learning based Hashing for cross-modal retrieval (APLH) is proposed, which uses adversarial training to learn low-dimensional features from different modalities and to ensure the distribution consistency of low-dimensional features across modalities. On this basis, cross-modal projection matching constrain (CMPM) is introduced which minimizes the Kullback-Leibler divergence between feature projection matching distributions and label projection matching distributions, and label information is used to align similarities between low-dimensional features of data with similarities in semantic space. Furthermore, in the Hashing learning phase, a weightedcosine triplet loss is introduced to further exploit the semantic information of the data, and to reduce the quantization loss, the Hashing function using a discrete optimization approach is optimized. The mean average precision of the proposed method on three databases MIRFlickr25K, NUS-WIDE and Wikipedia is better than other methods of comparison, which verifies the effectiveness of CMPM and shows the robustness of our method.