Robust human gesture recognition by leveraging multi-scale feature fusion

Abstract

In the state-of-the-art human–computer interaction (HCI) systems, gestures feature is more intuitive, natural and, easy-to-acquire than the other visual features. Due to the high descriptiveness of hand movements and the scarcity of labeled human gesture samples, gesture recognition models is prone to overfitting in practice. In order to handle this problem, to facilitate the model parameter learning during training, this paper introduces a disturbing IoU strategy to alleviate overfitting during the training stage from a ROI discriminative network. Moreover, during the deep gesture feature extraction, feature maps with the same size output by different layers are intelligently fused. This strategy can effectively preserve the key information in a small scale, reduce the information loss, and improve the generalization ability of the high-level gesture features. Comprehensive experimental results on the extended VIVA dataset and the NTU dataset have shown that the proposed model achieves a better mAP performance than its competitors