Depth Based Static Hand Gesture Segmentation and Recognition

Abstract

AbstractHand Gesture Recognition (HGR) refers to identifying various hand postures which helps in nonverbal communication with humans or machines. It also finds various applications in the area of Human Computer Interaction (HCI) like robotics control and Sign Language Recognition (SLR) for communication with specially abled people. Classification of hand gestures in varying lighting conditions and occlusion is still a challenging task. In this paper, a two-stage approach is proposed where hand region is segmented in first stage followed by classification of segmented hand gestures in the second stage. In the segmentation stage, a novel hybrid approach of coarse to fine hand segmentation is proposed where YOLO (You Only Look Once) network is used to detect the hand region at coarse which is further refined using Grabcut algorithm to obtain the fine boundary of hand region. In classification stage, hand segmented RGB and depth image are combined as 4-channel RGB-D input to the classification CNN model. Proposed method was evaluated on the OUHANDS dataset and achieved the validation accuracy of 98.75% and test accuracy of 86.50%.KeywordsCNNDepthHand gesture recognitionRGB-DYOLOGrab CutSegmentation