Intention-focused active reranking for image object retrieval

Abstract

We consider the problem of ranking refinement for image object retrieval, whose goal is to improve an existing ranking function by a small number of labeled instances. To retrieve the relevant image object, one state-of-the-art approach is to use the relevance feedback: it first ranks the images in database based on a given ranking function (i.e., base ranker), and then rerank the initial result by further introducing user's feedback information. The key challenge of combining the information from the base ranker and user's feedback comes from the fact that the base ranker tends to give an imperfect result and the information obtained from user's feedback tends to be very noisy. This paper describes an Intention-Focused Active Reranking, an approach for automatically finding the right information to re-estimate the query model. Three novel strategies are proposed to boost the performance of the base ranker: (1) an active selection criterion, which obtains a small number of feedback images that are the most informative to the base ranker for user labeling; (2) the user intention verification, which captures the user's intention in object level to alleviate the query drift problem; (3) a discriminative query model re-estimation, which augments the generative approach with a model of the discriminative information conveyed by positive and negative feedback information. Experiments on a real world data set demonstrate the effectiveness of the proposed approach and furthermore it significantly outperforms the baseline visual bag-of-words retrieval.