Recognition of Sign Language by Using the Implementation of Convolutional Neural Networks

Abstract

Abstract: The discipline of Sign Language Recognition (SLR) is primarily concerned with the development of techniques that may translate sign language into written or spoken language. The primary purpose of this field is to make it easier for people who are deaf-mute to communicate with the general public. Despite the inherent difficulties that are brought about by the activity's intricacy and the enormous repertory of manual movements that it demands, this specific activity has a significant influence on the larger social setting. The approaches that are now in use for sign language recognition (SLR) are dependent on features that are developed manually in order to portray the motion of sign language. These features are then used in the process of developing classification models. On the other hand, the issue of building dependable features that are capable of properly responding to the wide variety of hand gestures is a considerable difficulty. In this investigation, we suggest a unique convolutional neural network (CNN) architecture as a possible solution to the issue that was described before. One of the capabilities of the Convolutional Neural Network (CNN) is its capacity to independently extract discrete spatial-temporal properties from unprocessed video streams. This eliminates the need for previous information and eliminates the demand for feature building. CNN utilizes many video feeds that include color information, depth cues, and bodily joint locations. These streams are included into the network. A successful integration of color, depth, and trajectory information is achieved by the use of these streams as input. The effectiveness of the Convolutional Neural Network (CNN) is going to be improved as a result of this activity. Through the application of the proposed model to a real dataset received from Microsoft Kinect, the validity of the model is shown. The results of our investigation indicate that its performance is better to that of conventional techniques that are dependent on attributes that are manually developed.