Learning discriminative CNN features and similarity metrics for image retrieval

Abstract

This paper proposes a CNN-based retrieval framework that uses Siamese network to learn a CNN model for image feature extraction. Model training and testing stages often use the same similarity metric. But this paper adopts a contrastive loss function with different distance metrics to fine-tune a pre-trained CNN model, and applies different distance metrics in testing stage. Through experimenting with different similarity metrics, this paper finally finds that using the L2 distance to specify the contrastive loss function while applying cosine similarity during testing achieves the best performance. Its mean average precision (mAP) achieves 58.2%, and is 1–2% higher than the previous best method [4]. It shows that the similarity metrics for training and testing need not be same. Subsequently, we learn more generalized similarity metrics for model training and testing independently by minimizing a hinge loss function defined over a pair of global image representations. Compared with the existing image retrieval methods that involve hand-crafted features, our proposed framework performs well on four typical retrieval datasets.