Un-supervised and semi-supervised hand segmentation in egocentric images with noisy label learning

Abstract

With the rapid development of wearable devices and technologies, hand segmentation remains a less explored direction in egocentric vision, which is very important for activity recognition, rehabilitation, robot self-learning, etc. To overcome the high cost of auxiliary equipment and pixel-level annotations, we present an un-supervised hand segmentation method for egocentric images. Firstly, a fully convolutional neural network (FCN) is pre-trained in source dataset containing pixel-level annotations. Then, in target dataset without labels, the network is re-trained with optimized masks, which are produced by modified local and global consistency learning (LLGC) based on pre-segmentation and superpixel features. Finally, hand segmentation is realized in an alternative way. Furthermore, to balance segmentation accuracy and the cost on labeling, we propose a new semi-supervised image segmentation framework with three sub-nets based on the optimized noisy masks and a small number of clean labeled data. Experimental results in two target datasets indicate that the proposed methods could achieve better performance than other methods.