Gesture Recognition Using Wearable Sensors With Bi-Long Short-Term Memory Convolutional Neural Networks

Abstract

In this study, we propose a gesture recognition system that is applicable for controlling home appliances. We utilized sensors embedded inside common smart watches, such as accelerometers and gyroscopes, for alleviating the obtrusiveness to users. One-dimensional convolutional neural networks and bi-long short-term memory (1D-CNN-biLSTM) are proposed for analyzing, learning, and representing features from the sensor signals. In addition, a dataset of 18,000 gestures with 18 labels was collected from 20 subjects to verify our proposed methods. Notably, the proposed hand gesture vocabulary was found to be easy to learn for users. Moreover, it provides them with improved control over their home appliances. The results of an empirical experiment conducted on three public datasets in addition to our self-collected dataset (GesHome) indicate that the proposed 1D-CNN-biLSTM model can achieve an F1-score of 90% and outperforms previous state-of-the art methods. Moreover, a demonstrative system was employed to illustrate the efficiency of the proposed model for home appliance control in a real-world scenario.