Recognition of Indian Sign Language Using ORB with Bag of Visual Words by Kinect Sensor

Abstract

ABSTRACTRecognition of Indian Sign Language (ISL) could bridge the gap between deaf-mute people and society. Hand recognition is a key requirement for ISL recognition system. In this paper, the hand region is segmented from the depth image using the Microsoft Kinect Sensor in the cluttered environment. The depth image obtained is then used to implement supervised machine learning by extracting and training the features of images. Here, by comparing various methods, it is depicted that ORB (Oriented FAST and Rotated BRIEF) outruns others in terms of accuracy. ORB is invariant to scale, rotation, and lighting conditions. ORB is also fused with various classification techniques to gain the optimum result. The method is applied to images of ISL 0–9 and is also compared with some standard datasets. Tuning of the ORB with k-NN classification produces an average recognition accuracy of 93.26% with ISL dataset.