Explainable federated learning for privacy-preserving bangla sign language detection

Abstract

Sign language plays a crucial role as an inclusive mode of communication for individuals experiencing challenges addressing both hearing and speech aspects. Bengali Sign Language serves as a lifeline for those with hearing or speech impairments, enabling effective communication. However, the intricacies of sign language, especially Bengali Sign Language, present unique challenges. Traditional methods of sign language recognition and interpretation involve privacy concerns as they rely on central servers. However, the research on identifying Bengali signs from hand gestures is still largely unexplored, and to the best of current knowledge, no research specifically focuses on safeguarding privacy in this aspect. Therefore, this research introduces a groundbreaking approach utilizing federated learning (FL), ensuring user privacy while classifying and detecting Bengali Signs from hand gestures. After experimenting with various averaging algorithms, employing Visual Geometry Group (VGG19) with Federated averaging (FedAVG) yielded outstanding results. With the utilization of FedAVG, this system achieved a remarkable accuracy rate of 98.36% with the VGG19 model, representing a significant advancement in the realms of sign language interpretation. This research makes substantial contributions to areas related to sign language detection and privacy-aware machine learning techniques, benefiting the deaf community and society as a whole.