Application of Artificial Neural Network on South African Sign Language Recognition System

Abstract

Sign language assumes a significant part in filling the oral communication void for non-hearing or hard of hearing people. Despite the use of sign language, most people might not understand sign language clearly. Globally, research has been ongoing to explore and recommend various sign language recognition systems for different sign languages. On the other hand, researchers are still faced with challenges of practically implementing recognition systems. Though, machine learning has emerged as one of the advanced sign language recognition systems. Artificial neural network (ANN) models and techniques are machine learning models which are widely utilised to improve the accuracy level thus offering better results when recognising sign language gestures. However, there are few or no actively existing South African Sign Language (SASL) automatic recognition systems readily available to facilitate communication seamlessly and instantly between non-hearing and hearing people. This research concluded that application of finger tracking technique significantly improved feature extraction, training and prediction of hand gestures produced under different conditions, i.e. skin color and background. While the application of artificial neural network on SASL recognition system led to better performance with 98.77% and 97.5% accuracy scores during training and prediction, respectively. In this paper, the proposed ANN utilised finger tracking techniques and forward and backward algorithms to implement a system that recognizes fingerspelling in SASL.