A Machine Learning Based Sign Language Interpretation System for Communication with Deaf-mute People

Abstract

An utmost necessity of the deaf-mute people is to communicate with the non-mute people without the knowledge of sign language in different environments including office premises, shopping centers, educational institutions, and the like. Although several systems exist for teaching or learning sign language for the deaf-mute people, little attention has been paid to the development of useful and usable tools for effortless communication between mute and non-mute people. Therefore, the objective of this research is to develop an intelligent sign language interpretation system to connect with deaf-mute community, which will be used as a two way correspondence between speech impaired and regular speaking people. To attain this objective, firstly, a survey on recent vision-based gesture recognition approaches has been carried out. Secondly, an efficient and improved method for recognition of static hand posture and temporal gesture, focusing on finger-spelling method has been proposed which was developed using raspberry pi with the help of machine learning. Finally, the developed system was evaluated with 60 participants including the deaf-mute people and was found to be capable of successfully carrying out its functions with good performance.