Image feature ratio model: similarity measures with both semantic and visual features in interactive image retrieval

Abstract

The feature contrast model (FCM), which is the simplest form of the matching function in Tversky's set-theoretic similarity, is a famous similarity model in psychological society. Although FCM can be employed to explain the similarity with both semantic and perceptual features, it is very difficult for FCM to measure natural image similarity with semantic features because of the requirement that all features must be binary and the complex mechanism that semantic features are transformed into binary features. The fuzzy feature contrast model (FFCM) is an extension of FCM, which replaces the complex feature representation mechanism with a proper fuzzy membership function. By this fuzzy logic, visual features, in the FFCM, can be represented as multidimensional points instead of expansible feature set and used to measure visual similarity between two images. Based on the analysis of the distinction between two feature structures (i.e., the expansible feature set and multidimensional vector), we propose a ratio model, which expresses similarity between two images as a ratio of the measures of semantic features set to that of multidimensional visual features. Experiments results, over real-world image collections, show that our model addresses the distinction between semantic and visual feature structures to some extension. In particular, our model is suit for the case that semantic features are implicitly obtained from interaction with users and the visual features are transparent for users, for example, the relevance feedback in interactive image retrieval.