UTILIZING CONVOLUTIONAL NEURAL NETWORKS TO COMPREHEND SIGN LANGUAGE AND RECOGNIZE EMOTIONS

Abstract

The inability to communicate verbally is widely acknowledged as a significant disability. The primary objective of this research is to create a practical system aimed at individuals with hearing impairments, particularly those who depend on sign language as their primary means of communication. This system aims to enable deaf individuals to express themselves, communicate effectively and facilitate understanding of their language which would otherwise be challenging since most people are unfamiliar with sign language. By employing human gesture interpretation and motion capture, this technology can facilitate the translation of sign language into spoken language and vice versa. Despite the existence of various methods to convert sign language into voice, none of them currently provide an entirely intuitive user interface. Our objective is to create a system that not only translates sign language but also integrates a natural user interface, thus enhancing accessibility for individuals who are blind or have visual impairments. This system will achieve this by recognizing facial expressions and effectively conveying emotions behind words assisting visually impaired individuals in expressing themselves more effectively. Hence, in this project, we tentatively aim to build a system that can ease the lives of blind, deaf and dumb people to some extent. Normal communication like normal people do might never be possible for differently abled people but through our project, we try to provide them with a tool that can help them experience normalcy while communicating with normal people.