Deep continual hashing for real-world multi-label image retrieval

Abstract

Deep Hashing has achieved great success in large-scale image retrieval due to binary code’s storage and computation efficiency. However, its learning paradigm under real-world environments is less studied, and most existing approaches are developed in the closed scenario, e.g., simple and unchanging semantics. When images of new classes emerge, they have to retrain the model on all history training datasets, but the constant data uploading makes this impractical. This paper proposes a novel method, called continual deep semantic hashing (CDSH), for learning binary codes of multi-label images with increasing classes. The CDSH consists of two hashing networks. One learns to hash the increasing semantics of data, i.e., label, into the semantic codes and accumulates label-code pairs as long-term knowledge, incorporating empirically verified loss and designed special regularization to ensure encoding old labels unchanged. The other learns to map images to the corresponding semantic code from a probabilistic view and solid knowledge via retaining history exemplar samples and projecting model gradients. We theoretically prove this improves the probability of old data’s code unchanged after the model is updated. Extensive experiments on four widely used image datasets demonstrate that the CDSH method can continually learn hash functions and yield state-of-the-art retrieval performance.