Machine Learning-Based Gesture Recognition for Communication with the Deaf and Dumb

Abstract

A deep learning model specifically designed to recognize signs in sign language is the foundation of the Sign Language Recognition system. Sign Language is a visual language used by the deaf and hard of hearing community to communicate with one another and the general public. Sign language is a kind of nonverbal communication based on the use of hand gestures. The ability to communicate socially and emotionally is greatly aided when the speech and hearing challenged have access to sign language. The model developed in this paper captures the images through live web cam and displays the sign language meaning on the screen as text output. The model is trained and built by deep learning framework using Convolution Neural Networks (CNN) in this work. The model is trained with images of hand gestures captured through webcam using Computer Vision and then after successful training, the system performs recognition process through matching parameters for a given input gesture and finally displays the sign language meaning of the gesture as text output on the screen.