Detection of Hands for Hand-Controlled Skyfall Game in Real Time Using CNN

Abstract

With human-computer interaction technology evolving, direct use of the hand as an input device is of wide attraction. Recently, object detection methods using CNN models have significantly improved the accuracy of hand detection. This paper focuses on creating a hand-controlled web-based skyfall game by building a real time hand detection using CNN-based technique. A CNN network, which uses a MobileNet as the feature extractor along with the single shot detector framework, is used to achieve a robust and fast detection of hand location and tracking. Along with detection and tracking of hand, skyfall game has been designed to play using hand in real time with tensor flow framework. This way of designing the game where hand is used as input to control the paddle of skyfall game improved the player interaction and interest towards playing the game. This model of CNN network used egohands dataset for detecting and tracking the hands in real time and produced an average accuracy of 0.9 for open hands and 0.6 for closed hands which in turn improved player and game interactions.