Adaptive all-season image tag ranking by saliency-driven image pre-classification

Abstract

Social image tag ranking has emerged as an important research topic due to its application on web image search. This paper presents an adaptive all-season tag ranking algorithm which can handle the images with and without distinct object(s) using different tag ranking strategies. Firstly, based on saliency map derived from the visual attention model, a linear SVM is trained to pre-classify an image as attentive or non-attentive category by using the gray histogram descriptor on the corresponding saliency map. Then, an image with distinct object is processed by the tag saliency ranking algorithm emphasizing distinct object, which combines image saliency map with sparse representation based multi-instance learning algorithm. On the other hand, an image without distinct object can be processed by the tag relevance ranking algorithm via the sparse representation based neighbor-voting strategy. Such adaptive all-season tag ranking strategy can be regarded as taking full advantage of existing tag ranking paradigms. Experiments conducted on well-known image data sets demonstrate the effectiveness of the proposed framework.