A Real Time Vision System Based on Deep Learning for Gesture Based Human Machine Interaction

Abstract

AbstractThe use of gestures is one of the principal ways of communication among human beings when other forms, such as speech, are not possible. Taking this as a basis, the use of gestures has become also a principal form of human machine interaction in many different fields, ranging from advanced industrial setups where robots are commanded by gestures, to the use of hands to remotely control multimedia devices present at home.The majority of the systems for gesture detection are based on computer vision, either color images, depth images or point clouds, and have to overcome the inherent problems of image processing: light variations, occlusions or change of color. To overcome all these problems, recent developments using deep learning techniques have been presented, using Convolutional Neural Networks.This work presents a hand gesture recognition system based on Convolutional Neural Networks and RGB images that is robust against environmental variations, fast enough to be considered real time in embedded interaction applications, and that overcomes the principal drawbacks of the state of the art hand gesture recognition systems presented in previous works.KeywordsReal timeDeep learningGesture detectionEmbedded systems