Smart Gesture-based Control in Human Computer Interaction Applications for Special-need People

Abstract

Gesture recognition has been recognized as a natural way for the communication especially for elder or impaired people. Hand gesture recognition is an important research issue in the field of human-computer interaction (HCI), because of its extensive applications in virtual reality, sign language recognition, and computer games. There is a number of algorithms addressing different aspects of the Gesture recognition problem have been proposed. While image-based techniques have been widely studied, it may be affected by lighting conditions, large variations of the hand gesture and textures. Recently, with the developments of new technologies and the large availability of inexpensive depth sensors, real time gesture recognition has been faced by using depth information and avoiding the limitations due to complex background and lighting situations. This paper introduces an enhanced automated model for hand gesture recognition using convolution neural network (CNN). In this paper, a hand gesture recognition model with Kinect sensor has been proposed, which operates robustly in uncontrolled environments and is insensitive to hand variations and distortions. The proposed model uses both the depth and color information from Kinect sensor to detect the hand shape, which ensures the robustness in cluttered environments. The proposed model consists of two major modules, namely, hand detection and gesture recognition. Experiments have been conducted on large dataset to demonstrate the efficiency of the proposed model. The experimental results show an outstanding performance in the terms of accuracy, recall and precision.