DenseAttentionSeg: Segment hands from interacted objects using depth input

Abstract

Hand segmentation is an important task in computer vision, which is usually the foundation of hand pose recognition, hand tracking, and reconstruction. For hand segmentation, it is more challenging when the hand is interacting with objects, but handling interacting motions is more important for applications like HCI and VR. In this paper, we propose a real-time DNN-based technique to segment hand and object in interacting motions from a single depth input. Our model is called DenseAttentionSeg, which contains a dense attention mechanism which effectively fuses information in different scales and improves the quality of result with skip-connections. Besides, we introduce a contour loss in model training, which helps to generate accurate hand and object boundaries. Finally, we propose our InterSegHands dataset, a fine-scale hand segmentation dataset containing about 52k depth maps of hand-object interactions, with the ground truth segmentation masks. Our experiments evaluate the effectiveness of our techniques and datasets, and indicate that our method outperforms the current state-of-the-art deep segmentation methods in handling hand-object interactions.