Perceptual color and spatial texture features for segmentation

Abstract

We develop spatially adaptive, low-level, color and spatial texture features based on perceptual principles about the processing of texture and color information. We then propose an algorithm that combines these features to obtain image segmentations that convey semantic information that can be used for content-based retrieval. Our focus is images of natural scenes. The color texture features are based on the estimation of spatially adaptive dominant colors, which on one hand, reflect the fact that the human visual system cannot simultaneously perceive a large number of colors, and on the other, the fact that image colors are spatially varying. The spatially adaptive dominant colors are obtained using a previously developed adaptive clustering algorithm for color segmentation. The spatial texture features are based on a steerable filter decomposition, which offers an efficient and flexible approximation of early processing in the human visual system. We use the local energy of the subband coefficients as a simple but effective characterization of spatial texture. A median filter is used to distinguish the energy due to region boundaries from the energy of the textures themselves. Texture feature estimation requires a finite neighborhood that limits spatial resolution, while color segmentation provides accurate and precise edge localization. By combining texture with color information, the proposed algorithm can obtain robust segmentations that are accurate and precise. The performance of the proposed algorithm is demonstrated in the domain of photographic images, including low resolution, degraded, and compressed images.