A Weighted Deep Ensemble for Indian Sign Language Recognition

Abstract

This work concentrates on developing an Indian sign language (ISL) recognition system using a forearm-worn wearable device to assist hearing-impaired persons. A novel ensemble of convolution neural networks (CNN) is proposed for robust ISL recognition using multi-sensor data. The accuracy for classification of 50 ISL signs improved from 92.5% obtained using a single CNN to 94.2% with 10 ensemble members created using the bagging approach and soft-voting for decision aggregation. Then, the ensemble of CNNs was optimized using weighted voting, where the weights were determined using a differential evolution algorithm. This further improved the classification accuracy to 96.6% with 10 ensemble members.