Matching and retrieval based on the vocabulary and grammar of color patterns

Abstract

While it is recognized that images are described through color, texture and shapes of objects in the scene, general image understanding is still difficult. Thus, to perform image retrieval in a human-like manner one has to choose a specific domain, understand how users achieve similarity within that domain and then build a system that duplicates human performance. Since color and texture are fundamental aspects of human perception we propose a set of techniques for retrieval of color patterns. To determine how humans judge similarity of color patterns we performed a subjective study. Based on the results of the study five most relevant visual categories for the perception of pattern similarity were identified. We also determined the hierarchy of rules governing the use of these categories. Based on these results we designed a system which accepts one or more texture images as input, and depending on the query, produces a set of choices that follow human behavior in pattern matching. Processing steps in our model follow those of the human visual system, resulting in perceptually based features and distance measures. As expected, search results closely correlate with human choices.