Expressive Sign Language Recognition using CNN

Abstract

A major shortcoming in our society is a social barrier between the differently abled members of the society and the abled folks. One of the most important aspects of human beings, being regarded as social animals, is in fact communication. Communication is also a major obstacle faced by the hearing and vocal disabilities people. This inability to communicate leads to frequent problems and hinders the daily activities of a person with hearing and vocal disabilities. The underlying reason for this disparity is that abled folks don’t learn and aren’t taught Sign Language which is the main means of communication for a person with hearing and vocal disabilities. Thus, abled folks are incapable of having a normally fluent conversation with these different sections of the society. Consequently, in a verbal exchange among hearing and speech impaired individuals and an able person the convenience of communique and consequently the consolation degree is hampered. So, in our project, we have proposed a cost-efficient solution to overcome this communication barrier. This solution can be easily used by everyone and can also be, with some modifications, made to work on most platforms which have a camera module. Our approach uses the integrated camera module to capture real time hand gestures based on hand key points or landmarks and the algorithm using machine learning techniques, displays the alphabet that the gesture is representing. KEY WORDS: Convolution Neural Network, Transfer Learning, Deep Learning, Artificial Neural Network, Support Vector Machines