Multi-task Learning for Deep Semantic Hashing

Abstract

Deep learning to hash has emerged as a popular technique for large-scale image retrieval. Existing deep learning to hash methods seek to solve the single retrieval task within one stream framework or jointly solve the retrieval task and the classification task within two stream framework. Consequently, the semantic information is not fully exploited to generate compact and discriminative hash codes. In this paper, we propose a multi-task learning architecture for deep semantic hashing (MLDH), which incorporates the retrieval task and the classification task within one-stream framework. Specifically, we introduce a COCO loss to learn compact binary codes for the classification task. For the retrieval task, we introduce a pairwise loss to learn discriminative binary codes. Finally, these two tasks are investigated into one-stream deep learning framework. Extensive experiments show that MLDH can outperform state-of-the-art methods on benchmark datasets.