AI based Two Way Sign Language System

Abstract

Most hearing persons do not understand sign language. But hearing-impaired people communicate through the sign language only. Hence, there is a challenge in the communication between hearing and the hearing-impaired people. The proposed system is to build a bridge between the hearing and hearing-impaired communities and to start two-way communication. The proposed system suggests a two-way sign language translator that translates speech in real time into sign language and vice versa. Using Python’s OpenCV Library, the proposed system processes the video frame by frame. Additionally, each frame’s background is removed using the Background/Foreground Segmentation Algorithm based on Gaussian Mixtures. The outlines of the processed image are classified into the comparable written language terms using a convolutional neural network (CNN). In order to accurately preserve the sign language’s grammar during conversion from sign language to voice, we use gTTS (Google Text-to-Speech) and fundamental NLP. Enrolment and reading rates among hearing-impaired youngsters are significantly lower than the general population average. This technology will therefore assist regular schools in better integrating the hearing challenged community, making education more accessible and affordable for them.