Segmentation-Based Background-Inference and Small-Person Pose Estimation

Abstract

Despite encouraging results have been achieved in human pose estimation in recent years, it remains challenging problems. When the background is similar to the human body parts, and there are small persons with low-resolution in the image, the performance may degrade dramatically. This paper addresses problems in background-inference and small-person pose estimation. To achieve this, a novel pose estimation algorithm is proposed on the basis of person semantic segmentation deep neural network. Different from most previous methods with a single pose estimation model, we generate mixture models with pose estimation and semantic segmentation. We introduce novel generative adversarial model and auxiliary model to realize the semantic segmentation network, which can handle the confusion of the similar regions in the background. In addition, to address the problem of the scale differences between big and small persons’ keypoints, we add additional position and channel attention modules to the first two stages of OpenPose. We conduct extensive experiments on COCO and VOC datasets. And we compare the proposed method with the most popular state-of-the-art human pose estimation and semantic segmentation frameworks, including MultiPoseNet, Deterton2 and DeepLab V3. Our experimental results show that the proposed method is more accurate than the state-of-the-art algorithms and performs effectively in tackling the complex situations.