Learning discrete class-specific prototypes for deep semantic hashing

Abstract

Deep supervised hashing methods have become popular for large-scale image retrieval tasks. Recently, some deep supervised hashing methods have utilized the semantic clustering of hash codes to improve their semantic discriminative ability and polymerization. However, there exists a semantic gap between the hash codes learned from the visual features and the semantic labels, which weakens the generalization ability of these methods. In addition, the manifold structure of the hash codes in the Hamming space is ignored. In this paper, we propose a novel deep semantic hashing method by learning discrete class-specific prototypes (DCPH). Specifically, we utilize the label information to learn discrete class-specific prototypes as the intermediate semantic representations of the semantic labels, which can reduce the semantic gap between the semantic labels and the hash codes and improve the correlation between the class-specific prototypes and the hash codes. Subsequently, we construct a bipartite graph to build coarse semantic neighborhood relationship between the hash codes and the class-specific prototypes, which can preserve the manifold structural information. Moreover, we utilize the pairwise supervised information to construct a fine semantic neighborhood relationship between the hash codes. Finally, we propose a novel hashing loss based on multitask learning framework to incorporate them into an end-to-end one-stream deep neural network architecture. Experimental results on several large-scale datasets demonstrate that the proposed method can outperform state-of-the-art hashing methods.