Integrating visual saliency and consistency for re-ranking image search results

Abstract

The paper investigates two mechanisms, visual consistency and visual saliency, in web image search: (1) In most current web image search engines, such as Google Image Search and Yahoo Image Search, the images that closely related to the search query are typically visually similar. These visually consistent images which occur most frequently in the first few web pages will be given higher ranks. (2) From visual aspect, it is obvious that salient images would be easier to catch users' eyes and more likely to be clicked than the cluttered ones in low-level vision. In addition, we also observe the fact that the visually salient images in the front pages are often relevant to the user's query. The principal novelty of this paper is in combining visual saliency and consistency to re-rank the results from search engines to make the re-ranked images more satisfying in both vision and content. The experimental results on a real world web image dataset demonstrate that our approach can effectively improve the performance of image retrieval.