Recognition of American sign language using LBG vector quantization

Abstract

Sign language is a widely used and accepted standard for communication by people with hearing and speaking impairments. A sign language is a language which uses visually transmitted sign patterns, instead of acoustically conveyed sound patterns, to deliver the meaning. Several ways to generalize the American sign language recognition have been proposed in the past [14] [15]. In this paper, novel method of American sign language recognition with Shape and Texture features has been proposed. Many of the existing systems require the person gesticulating to use special data acquisition devices like data gloves which are expensive and difficult to handle. We proposes a more flexible vision based approach where the person is free from additional equipment. The Shape features and Texture feature are more unique, so a novel technique based on combination of these is derived and proposed here. For extracting shape features standard gradient operator such as Robert, Prewitt, Sobel, Canny, Freichein, Kirsh and Laplace are used and for texture feature vector quantization techniques are used. The gradient mask images of the character images are obtained and then LBG vector quantization algorithm is applied on these gradient images to get the codebooks of various sizes. These obtained LBG codebooks are considered as shape texture feature vectors for American sign language recognition. The database includes 26 for American sign language alphabets taken by 12 different people. The images are saved in a jpeg file format and stored in separate folder. Thus there are total 312 images were use for our project and 8 code book sizes (from 4 to 512). The nearest neighbour (KNN) algorithm is considered as performance comparison criteria for proposed character recognition techniques. The best performance is observed in LBG for codebook size 8 of Canny operator and the next best is seen for codebook sizes 4 of freichein gradient mask for feature extraction. The LBG VQ [11] design algorithm is an iterative algorithm which requires an initial codebook C. This initial codebook is obtained by the splitting method. In this method, an initial code vector is set as the average of the entire training sequence. This code vector is then split into two. The iterative algorithm is run with these two vectors as the initial codebook. The final two code vectors are splitted into four and the process is repeated until the desired number of code vectors is obtained.