Convolutional Neural Network based Sign Language Recognition to Assist Online Assessment

Abstract

Sign languages are the visual mode of communication used by the deaf individuals. Learning sign languages at an early age is very essential for the social and intellectual developments of the deaf community. However, the lack of qualified trainers is a challenging problem that restricts the opportunity for getting formal sign language education for the deaf individuals. Hence, the need for developing an online system for learning sign language gestures through self assessment is very essential. The primary requirement for developing such a system is an efficient sign language recognition (SLR) model. This paper presents an efficient convolutional neural network (CNN) based model for automatically recognizing fingerspellings in sign languages. The model has been tested on a novel Indian sign language (ISL) fingerspelling dataset as well as a publicly available hand posture dataset, and has obtained promising results.