Gesture Recognition Based on Dual-channel Feature Fusion

Abstract

In order to solve the problems of traditional gesture recognition based on artificial feature extraction, such as complicated operation, low accuracy, and insufficient feature extraction of classic convolutional neural network, we propose a two-stage dual-channel convolutional neural network. The entire network structure is divided into two stages which including a segmentation stage and a recognition stage. In the segmentation stage, a network consisting of a fully convolutional residual network and a Pyramid Pooling Module (PPM) module are used to execute the segmentation. As the shown in the experimental verification results, the proposed segmentation network designed has a very good effect in light changes and complex environments; For the recognition stage, we use a network which is composed of a two-channel convolutional neural network. The input of two channels is the RGB image and the feature map segmented by the segmentation network, and the output is the fusion of the extracted features of the two channels. For experimental verification, we paper use the OUHANDS data set. In theory, the recognition accuracy rate reaches 97.31%, and the network size and parameters are several times smaller than those of classic networks such as Lenet-5, ResNet-18, and VGGNet.