Deep Captioning Hashing Network for Complex Scene Image Retrieval

Abstract

Hashing methods have been widely applied to approximate nearest neighbor search for large-scale image retrieval, due to its computation efficiency and retrieval quality. Deep hashing can improve the retrieval quality by representation learning and hash coding. Existing deep hashing methods only take image spatial features into account and result in the lack of accurate semantic similarities of images pairs. In this paper, a novel deep hashing network, Deep Captioning Hashing Network (DCHN), is proposed to enhance semantic similarities of hash codes. In DCHN, the binary hash codes are generated in a Bayesian learning framework by fusing deep spatial representation and deep content captioning representation obtained by image captioning. Our analysis and simulation results have demonstrated that DCHN can achieve better retrieval performance in complex scene images compared with other supervised hashing methods and unsupervised methods on two complex scene image datasets MS COCO and NUS-WIDE.