Combining Hand Detection and Gesture Recognition Algorithms for Minimizing Computational Cost

Abstract

Hand gesture recognition is very important in human-computer interactions (HCI). The most common way to build a recognition system is to use a pre-trained convolution neural network. Relatively new architectures called convolution pose machine can represent a skeleton model of a hand or body from an image with sufficiently high accuracy. However, systems based on these architectures require valuable computational resources which might be inaccessible in practice. Convolutional layers of neural networks take a significant part of computer resources even if the target object (hand) is absent in the frame. This paper proposes a possible solution to this problem. It presents a combined hand gesture recognition system that uses a hand detector to detect hand in the frame and then switches to gesture classifier if a hand was detected. The paper illustrates the proposed combined algorithm. Descriptions of used hand detector and gesture recognition algorithms also are given. Equations for the evaluation of potential performance increase and experimental results are presented. The proposed system is tested on publicly accessible gesture bases and on video sequences prepared by the authors. The experimental results are consistent with theoretical estimates and demonstrate the benefits of the proposed gesture recognition system design.