Hand Gesture Recognition Using Faster R-CNN Inception V2 Model

Abstract

The real-time hand motion recognition under unconstrained environment is a challenging computer vision problem. The change in illumination and non-uniform background condition makes it very difficult to perform real-time hand gesture recognition operations. This paper demonstrates a region-based convolutional neural network for real-time hand gesture recognition. The custom dataset is captured under unconstrained environments. The Faster region-based convolutional neural network (Faster-RCNN) with Inception V2 architecture is used to extract the features from the proposed region. The average precision, average recall, and F1-score are analyzed by training the model with a learning rate of 0.0002 for Adaptive Moment Estimation (ADAM) and Momentum optimizer, 0.004 for RMSprop optimizer. The ADAM optimization algorithm resulted in better precision, recall and F1-score values after evaluating custom test data. For ADAM optimizer with intersection over union (IoU) =0.5:0.95, the observed average precision is 0.794, average recall is 0.833, and the F1-score is 0.813. For an IoU of 0.5, ADAM optimizer resulted in 0.991 average precision with a prediction time of 137ms.