Local image representations using pruned salient points with applications to CBIR

Abstract

Salient points are locations in an image where there is a significant variation with respect to a chosen image feature. Since the set of salient points in an image capture important local characteristics of that image, they can form the basis of a good image representation for content-based image retrieval (CBIR). The features for a salient point should represent the local characteristic of that point so that the similarity between features indicates the similarity between the salient points. Traditional uses of salient points for CBIR assign features to a salient point based on the image features of all pixels in a window around that point. However, since salient points are often on the boundary of objects, the features assigned to a salient point often involve pixels from different objects. In this paper, we propose a CBIR system that uses a novel salient point method that both reduces the number of salient points using a segmentation as a filter, and also improves the representation so that it is a more faithful representation of a single object (or portion of an object) that includes information about its surroundings. We also introduce an improved Expectation Maximization-Diverse Density (EM-DD) based multiple-instance learning algorithm. Experimental results show that our CBIR techniques improve retrieval performance by 5%-11% as compared with current methods.