Interactive Image Retrieval Using a Hybrid Visual and Conceptual Content Representation

Abstract

Many image databases available today have keyword annotations associated with images. State of the art low-level visual features reflect well the "physical" content and thus the visual similarity between images, but retrieval based on visual features alone is subject to the semantic gap. Alternatively, text annotations can be linked to image context or semantic interpretation but are not necessarily related to the visual appearance of the images. Keywords and visual features thus provide complementary information regarding the images. Combining these two sources of information is an advantage in many retrieval applications and recent work in this area reflects this interest.We introduce here a new feature vector, based on the keyword annotations available for an image database and making use of the conceptual information extracted from an external knowledge database. We evaluate the joint use of the proposed conceptual feature vector and the low level visual features both in a Query By Example (QBE) context and with SVM-based Relevance Feedback (RF). Our experiments show that the use of the conceptual feature vectors can significantly improve the effectiveness of both retrieval approaches.