Multi-modal Image Retrieval for Search-Based Image Annotation with RF

Abstract

Search-based annotation methods can be used for proposing descriptive keywords to users who need to annotate images e.g. in image stock databases. From the annotation output, users select keywords which they want to assign to the given image. The selected keywords can serve as a relevance feedback for additional annotation refinement. In this paper, we study the possibilities of exploiting the annotation relevance feedback, which is a novel problem that has not been systematically addressed yet. In particular, we focus on the subtask of utilizing the feedback for the retrieval of related annotated images that are subsequently used for mining of candidate keywords. We select three multi-modal search techniques that can be applied to this problem, implement them within a state-of-the-art search-based annotation system, and experimentally evaluate their usefulness for annotation quality improvement.