Gesture generation by the robotic hand for aiding speech and hard of hearing persons based on indian sign language

Abstract

Speech and hearing impairments are among the most common problems in Indian societies. It can affect anyone, whether children, adults, or more. Many different treatments can help to overcome hearing problems. Different types of hearing aids and cochlear implants help amplify sounds for better hearing. The type of language known as sign language is very scientific and has its grammar and syntax. Still, due to a need for more awareness among hard-of-hearing persons, they need to be made familiar with the institutions where they can learn and equip themselves for communication. This paper describes an approach to aid speech and hard-of-hearing persons so that they are free to communicate with persons who do not have speech and hearing disabilities based on the Indian Sign Language System. To find an appropriate solution, there is a need to develop a system that can act as an interpreter for speech and hard-of-hearing persons. The interpreter system is designed with the help of the Robotic hand model and is programmed using Raspberry Pi 4. Based on the experimental results, it can be observed that the robotic hands generated different signs of the alphabet corresponding to the speech commands uttered by an individual. Several experimental trials were conducted by ten persons who do not have any hearing disabilities. The results of the five experimental trials are shown in this paper. The estimation of performance parameters and statistical analysis are also carried out to analyze better and interpret the experimental results. Based on the experimental results, the proposed robotic hand interpreter system model accurately generates gestures corresponding to different alphabets used in the Indian Sign Language system, yielding an overall accuracy of 94 percent.