Intelligent real-time Arabic sign language classification using attention-based inception and BiLSTM

Abstract

Bio-inspired deep learning models have revolutionized sign language classification, achieving extraordinary accuracy and human-like video understanding. Recognition and classification of sign language videos in real-time are challenging because the duration and speed of each sign vary for different subjects, the background of videos is dynamic in most cases, and the classification result should be produced in real-time. This study proposes a model based on a convolution neural network (CNN) Inception model with an attention mechanism for extracting spatial features and Bi-LSTM (long short-term memory) for temporal feature extraction. The proposed model is tested on datasets with highly variable characteristics such as different clothing, variable lighting, and variable distance from the camera. Real-time classification achieves significant early detections while achieving performance comparable to the offline operation. The proposed model has fewer parameters, fewer deep learning layers, and requires significantly less processing time than state-of-the-art models.