Spatial pyramid deep hashing for large-scale image retrieval

Abstract

Effective feature representations and similarity measurements are crucial for large-scale image retrieval, and conventional methods often learn hash functions from a predefined hand-crafted feature space. Meanwhile, the spatial structure in raw images always lost in most previous methods. Encouraged by the recent advances in convolutional neural networks (CNNs), a novel Spatial Pyramid Deep Hashing (SPDH) algorithm is developed for the task of fast image retrieval. In our SPDH algorithm, the CNN with a spatial pyramid pooling and a locally-connected layer with binary activation functions is utilized to build the end-to-end relation between the raw image data and the binary hashing codes for fast indexing. Different from the fully-connected layer, the locally-connected layer can consider each local spatial bin as an independent unit and only connect the local bin to preserve the spatial pyramid structure for hash codes. The learning of both the hash function and the feature representations are jointly optimized via backward propagation with classification or similarity loss function on the large-scale labeled dataset such as ImageNet. Moreover, a spatial pyramid binary pattern matching algorithm is developed to achieve partial local similar matching among the images. Our experimental results have shown that our SPDH method can outperform several state-of-the-art hashing algorithms on the CIFAR-10, SIVAL and the Oxford buildings datasets.