KANSEI-BASED IMAGE RETRIEVAL WITH BAYESIAN DECISION MODELS AND RELEVANCE FEEDBACK

Abstract

The Kansei-based image retrieval (KBIR) aims to index images based on user's emotion and sensation. In this paper, we construct a KBIR system using scenery images as retrieval objects, which consists of three parts: visual features extraction, Kansei perception inference, and retrieval adjustment by relevance feedback. In the first part, low-level visual features such as color, texture, and shape features are extracted from perceptual viewpoints for all images. In addition, 5 pairs of Kansei keywords are selected to represent human Kansei perceptions from 5 different angles. In the second part, the multidimensional scaling technique is applied to discover the general relationship between image visual features and human Kansei perceptions, based on which we build a Bayesian decision model to characterize each image with 5 Kansei scores. The deduced Kansei evaluation represents the general impression or feeling engendered by an image and is used for the initial retrieval. In order to refine the retrieval results to make them agree with user's personal preference furthermore, we introduce the relevance feedback mechanism in our KBIR system and complete it with another Bayesian decision model. A prototype image retrieval system has been developed, the experimental results of which showed that the Bayesian decision models are promising for such kind of ambiguous retrieval.