Finger spelling recognition using neural network

Abstract

Finger spelling is a way of communication by expressing words using hand signs in order to ensure deaf and dumb community can communicate with others effectively. Therefore, a system that can understand finger spelling is needed. As a result of that, this work is conducted to primarily develop a tutoring system for finger spelling. To develop a robust real-time finger spelling tutoring system, it is necessary to ensure the accuracy of the finger spelling recognition. Even though there are existing solutions available for a decade, but most of them are just focusing on improving accuracy rate without implementing their solutions as a complete tutoring system for finger spelling. Consequently, it inspires this research project to develop a tutoring system for finger spelling. Microsoft Kinect sensor is used to acquire color images and depth images of the finger spells. Depth images are used to perform segmentation on the color images. After that, the segmented images are used as input and pass into a two hidden layers backpropagation neural network for classification.