Continuous Indian Sign Language Gesture Recognition and Sentence Formation

Abstract

Abstract Hand gestures are a strong medium of communication for hearing impaired society. It is helpful for establishing interaction between human and computer. In this paper we proposed a continuous Indian Sign Language (ISL) gesture recognition system where both the hands are used for performing any gesture. Recognizing a sign language gestures from continuous gestures is a very challenging research issue. We solved this problem using gradient based key frame extraction method. These key frames are helpful for splitting continuous sign language gestures into sequence of signs as well as for removing uninformative frames. After splitting of gestures each sign has been treated as an isolated gesture. Then features of pre-processed gestures are extracted using Orientation Histogram (OH) with Principal Component Analysis (PCA) is applied for reducing dimension of features obtained after OH. Experiments are performed on our own continuous ISL dataset which is created using canon EOS camera in Robotics and Artificial Intelligence laboratory (IIIT-A). Probes are tested using various types of classifiers like Euclidean distance, Correlation, Manhattan distance, city block distance etc. Comparative analysis of our proposed scheme is performed with various types of distance classifiers. From this analysis we found that the results obtained from Correlation and Euclidean distance gives better accuracy then other classifiers.