Supporting deafblind in congregational prayer using speech recognition and vibro-tactile stimuli

Abstract

Abstract Human behavior inference through smart pervasive technologies is a new research avenue. Sophistication in interaction modalities with self-customization based on context makes interaction easier in natural settings. Despite greater sophistication, disabled and elderly person still face difficulties in interaction with smart devices. The interaction generally depends on two or more sensory channels simultaneously, which is a big concern for persons with impaired vision and hearing. Person with dual sensory loss face problem in interaction with smart devices. Interaction can be effectively possible with this population group using alternative tactile sensory channel and vibro-tactile stimuli could be a great alternative for communication in natural settings to accommodate their needs. Feature-rich handheld and wearable smart devices could be the cost-effective substitute for assisting in ADL in natural settings for disabled and elderly person. Religious worships are important part of human life. By exploiting the advanced sensing technologies in Smartphone, an assistive prayer model is developed which is aimed at assisting deafblind person in semi-isolated natural settings, in a cost-effective manner which emphasizes on utilization of alternative sensory channels of a human. The model is validated through an App SmartPrayerAid aimed to assist deafblind Muslim in offering Islamic worship Namaz in congregational setting. Comprehensive evaluation of the App was conducted to prove the validity of the model and the aptness of the smart devices for assisting disabled person in natural settings. The evaluation was conducted with deafblind, non-deafblind and non-disabled subjects in controlled lab environment. Results indicated that speech recognition was achieved between 77.76% in real environment, while up to 96.29% in controlled lab environment using online speech recognition in Arabic. Effectiveness of Vibro-tactile stimuli perception using Smartphone on different body parts was also evaluated and accuracy in detection observed on front pocket of shirt was 84.29%, front pocket of the trousers was 92.14% and on upper arm was 52.43% and on wrist of the right hand using Smartwatch was observed 97.62%. The model can be utilized in other natural settings in similar situations after slight modification to assist a deafblind person in performing ADL and other social activities.