Attention Based Segmentation and Recognition Algorithm for Hand Postures Against Complex Backgrounds

Abstract

The Attention based Segmentation and Recognition (ASR) algorithm for hand postures against complex backgrounds is discussed in this chapter. The ASR algorithm can detect, segment and recognize multi-class hand postures. Visual attention, which is a cognitive process of selectively concentrating on a region of interest in visual field, helps humans to recognize objects in cluttered natural scenes. The ASR algorithm utilizes a Bayesian model of visual attention to generate a saliency map, and to detect and identify the hand region. Feature based visual attention is implemented using a combination of high level (shape, texture) and low level (color) image features. The shape and texture features are extracted from a skin similarity map, using a computational model of the ventral stream of visual cortex. The skin similarity map, which represents the similarity of each pixel to the human skin color in HSI color space, enhances the edges and shapes within the skin colored regions. The color features used are discretized chrominance components in HSI, YCbCr color spaces, and similarity-to-skin map. The hand postures are classified using shape and texture features, with a support vector machines classifier. The NUS hand posture dataset-II with 10 classes of complex background hand postures is utilized for testing the algorithm. The dataset contains hand postures from 40 subjects of different ethnicities. A total of 2,750 hand postures and 2,000 background images are available in the dataset. The hand postures vary in size and shape. The ASR algorithm is tested for hand detection and hand posture recognition using 10 fold cross-validation. The experimental results show that the algorithm has a person independent performance, and is reliable against variations in hand sizes and complex backgrounds.