DF-WiSLR: Device-Free Wi-Fi-based Sign Language Recognition

Abstract

Recent advancements in wireless technologies enable pervasive and device free gesture recognition that enable assisted living utilizing off the shelf commercial Wi-Fi devices. This paper proposes a Device-Free Wi-Fi-based Sign Language Recognition (DF-WiSLR) for recognizing 30 static and 19 dynamic sign gestures. The raw Channel State Information (CSI) acquired from the Wi-Fi device for 49 sign gestures, with a volunteer performing the sign gestures in home and office environments. The proposed system adopts machine learning classifiers such as SVM, KNN, RF, NB, and a deep learning classifier CNN, for measuring the gesture recognition accuracy. To address the practical limitation of building a voluminous dataset, DF-WiSLR augments the originally acquired CSI values with Additive White Gaussian Noise (AWGN). Higher-order cumulant features of orders 2, 3, and 4 are extracted from the original and augmented data, as the machine learning classifiers demand manual feature extraction. To reduce the computational complexity of machine learning classifiers, an informative and reduced optimal feature subset is selected using MIFS. Whilst the pre-processed original and augmented CSI values directly fed as input to an 8-layer deep CNN, it performs auto feature extraction and selection. DF-WiSLR reported better recognition accuracies with SVM for static and dynamic gestures in both home and office environments. SVM achieved 93.4% 98.8% and 98.9% accuracies in home and office environments respectively, for static gestures. For dynamic gestures, 92.3% recognition accuracy achieved in home environment. On augmented data, the corresponding gesture recognition accuracy values reported are 97.1%, 99.9%, 99.9%, and 98.5%.