Sign Language Recognition by Image Processing

Abstract

A prototype sign language interpreter that can converse in American Sign Language (ASL) is presented in this study. The visual stimulation of sign language aids in the development of speech and language. It promotes social contact, fosters the development of cognitive structures, and lessens harmful social behaviors. Early exposure to sign language supports language development in academic, social, and emotional domains. Building a user-hand gesture-only human-computer interface (HCI) is the foundation of the dependable paradigm. Using Python and OpenCV, hand gesture recognition is built using the theories of hand segmentation and the hand detection system. The training photos are labeled with different names based on what is in the database. A binary feature vector is created from each image. Convolutional neural networks (CNNs) and cameras are used to capture images from hand sign videos in real time, which are then used to carry out the feature extraction process. The algorithm generates output based on the projected maximum similarity between these features and the features of database photographs. With the aid of an application, this model provides adequate accuracy even in the absence of a constant or monochromatic background. The person's hand movements are captured on video with the camera; the video is subsequently processed and translated into text. The user hears the speech that has been transformed from the output. The suggested system recognizes letters, numbers, and certain common words in sign language. Raw photos and videos are transformed into corresponding text that can be read and comprehended by using image processing algorithms and neural networks to map the gesture to the right text in the training data. When paired with a sizable source database, this protocol will surely be highly beneficial for closing the communication gap between people who can talk and hear and those who cannot.