An Intelligent Sign Communication Machine for People Impaired with Hearing and Speaking Abilities

Abstract

People who are impaired with speaking and hearing abilities use sign language for communication between them, but it is a tough task for them to communicate with the outside world. Through this paper, we are proposing a system to convert Indian Sigh Language (ISL), American Sign Language (ISL) and British Sign Language (BSL) hand gestures to a textual format of the respective language as well as convert text in to their preferable Sign language. In this paper, we are capturing ISL, ASL, BSL gestures through a web camera. The streaming video of hand gestures is then sliced to distinct images to match the finger orientation to the corresponding alphabets. Finger orientations as features of the hand gestures in terms of angles made by fingers, numbers of fingers completely open, semi-open, fully closed, finger axis verticals or horizontal and recognition of each finger are prepossessed and required for gesture recognition. Implementation is done for alphabets uses single hand and results are explained. After prepossessing the hand part of the sliced frame in the form of masked image is projected to the extraction of features from the image frame. To classify different gestures we used SVM (Support Vector Machine), CNN (Convolutional Neural Network) for further testing the probable gesture and recording the accuracies of each algorithm. Implementation is done over our own regular ISL, BSL, ASL data-set made by us only, using the web camera of our laptops. Our Experimental results depict that our proposed work and methodology can work on different backgrounds like a background consist of different objects or may have some sort of color background etc. For text to sign conversion we create a video which tells respective text into sign language.