Semantic Constraints Matrix Factorization Hashing for cross-modal retrieval

Abstract

Cross-modal hashing methods have attracted considerable attention due to their low memory usage and high query speed in large-scale cross-modal retrieval. During the encoding process, there still remains two crucial bottlenecks: how to equip hash codes with cross-modal semantic information, and how to rapidly obtain hash codes. In this paper, we propose Semantic Constraints Matrix Factorization Hashing (SCMFH) which simultaneously considers modality-specific and cross-modal semantic information for hash codes with closed solution. Specifically, the original representation of each modality is factorized into modality-specific semantic representations, and then the cross-modal semantic similarity matrices is used to constrain correlation of modality-specific representation. Finally, all the modality-specific representations are regressed to the common hash codes, and the fast hash codes with a closed solution is obtained. Extensive experimental have been carried out on public datasets. The results show that the method outperforms many current cross-modal hashing methods in terms of mean average precision (mAP), up to 2.7%.