Image retrieval through qualitative representations over semantic features

Abstract

This paper proposes a qualitative knowledge-driven semantic modelling approach for image understanding and retrieval. The similarity measure is calculated for each query by exploiting the notion of conceptual neighbourhood--a measure of closeness between qualitative relations. The relative similarity of two images is proportional to the qualitative similarity measure value. The approach is motivated by the need to bridge the semantic gap between a human user and that of CBIR systems and enable semantic querying in such systems. Local semantic concepts, such as sky, grass, of an image are used to obtain a semantic image description. Four kinds of qualitative spatial representations have been applied to these semantic concepts. This allows for representation and reasoning of an image's content structures at a more abstract level than pixels or other low level features and provides a higher level, semantic basis for image understanding. We investigate whether such a representation of an image's visual content also provides an effective and natural way to provide content-oriented querying. We also investigate whether querying based on multiple representations is effective, and report on three voting schemes used to retrieve images in this way. The test data set having hand-assigned class labels has been used to have a metric evaluation of the retrieval accuracy. The results compare favourably with a non-qualitative representation based on the same semantic features which simply compares the percentage of each feature in pairs of images.