Semantic-consistent cross-modal hashing for large-scale image retrieval

Abstract

With an emphasis on saving storage and computation costs, hashing learning has got considerable success in cross-modal image retrieval. Most pioneer efforts in this direction either consider similarity across modalities or leverage discriminative information of the class labels to learn the common latent representation. However, the learnt representation only contains coherent semantics across modalities but could not be line with class-wise semantic structure. To attack this issue, we propose a semantic-consistent cross-modal hashing (SCCH) to take class semantic structure into consideration. It not only ensures the common representation to be consistent across modalities by directly learning the shared binary codes of samples via rotation transformation, but also restricts the class-wise representation line with the learnt binary codes. In this way, SCCH jointly preserves class semantic structure and avoids large quantization errors caused by the approximation of real values to binary codes. Moreover, we efficiently optimize SCCH via an iterative algorithm. Experiments on three publicly datasets demonstrate the superiority of SCCH against several representative start-of-the-art counterparts in light of performance metrics.