Re-ranking image search results by multiscale visual saliency model

Abstract

The paper focuses on two mechanisms, multiscale relevance and visual saliency, in web image search. First, in most current web image search engines, such as Google Image Search, Yahoo Image Search and so on, people judge the relevance of search results by the thumbnails and then click through the thumbnails to check if the corresponding image is really relevant. Basically the thumbnail and the corresponding image give the multiscale representations of the image. The second is that from visual point of view, it is obvious that salient images would be easier to catch users' eyes and more likely to be clicked than cluttered ones in low-level vision. In this paper, we build a multiscale saliency model and apply it to re-rank the results from web image search engines. Experimental results show that the model can achieve an average precision (AP) [1] of as high as 97%, and it improves the results of Google image search significantly.