Hand Gesture Recognition for Real-Time Game Play Using Background Elimination and Deep Convolution Neural Network

Abstract

AbstractIn recent scenarios, Gaming Technology is on the surge of the utmost extent of technical advancement. Contemporary gaming using game controllers limits its reach due to various economic issues. The game programmers are making efforts towards developing games without a game controller. The expensive game controllers restrict the usage of gaming among users. In this research work, a mobile-based human–computer interaction system has been developed which enables users to control video games with more physical interaction and without more restrictions. The novelty of work is to play games using any good quality mobile/web camera instead of the game controller. A Convolution Neural Network (CNN) is combined with background elimination to detect different hand gestures and further uses these gestures to control video games. In the first instance, a background elimination algorithm is used to extract the hand gesture image captured through a mobile/web camera. These hand images have been used to train as well as predict the type of gesture. CNN is used to detect gestures and to render the appropriate motion in the game. The designed human–computer interaction system concludes 98.2% accuracy of hand gesture recognition for considered hand gestures. The measured latency is also adequate while game play.KeywordsGesture recognitionConvolution neural networkGaming controlHuman–computer interactionComputer vision