Relevance feedback-based image retrieval interface incorporating region and feature saliency patterns as visualizable image similarity criteria

Abstract

Region and feature saliency (RFSP) is proposed as: (1) a new image similarity model; and (2) a new, visualizable representation of the image similarity criteria. RFSP, coupled with the proposed genetic-algorithm (GA)-based relevance feedback mechanism, is incorporated in the image retrieval interface. By capturing the two fundamental properties of the human visual system-region and feature saliencies-in a context-dependent sense, RFSP more accurately approximates the human similarity perception. By representing the image similarity criteria as a pattern of feature combinations distributed over the image regions, each having a different importance, RFSP enables the visualization-in a concise form-of the complex, low-level similarity criteria associated with each query image. None of the representative image similarity models captures both region and feature saliencies in a context-dependent sense. Furthermore, very few of the representative works-dealing with the relevance feedback in image retrieval-consider the visualization of the similarity criteria, as a user interface aspect. Also, this paper presents one of the first applications of GAs to the relevance feedback mechanism in the image retrieval field. The retrieval performance of the RFSP, coupled with the proposed GA-based relevance feedback mechanism, is evaluated on five test databases, with around 2500 images, covering 62 semantic categories. Compared with 11 of the representative image similarity models, including three which employ relevance feedback, RFSP brings in average between 6%-30% increase in the retrieval precision. The relevance feedback-based retrieval interface incorporating RFSP is demonstrated as well. Experiment results suggest that: (1) capturing the region and feature saliencies in a context-dependent sense improves the retrieval performance; whereas (2) visualizing the similarity criteria makes the relevance feedback-based image retrieval interface more user friendly, aiding the user in the understanding and expression of the information needs.