Complex Human Pose Estimation via Keypoints Association Constraint Network

Xuan Zhu,Peirong Chen,Rongzhi Wang,Jinye Peng,Zhenpeng Guo,Bin Li,Xin Liu

doi:10.1109/access.2020.3037736

Abstract

Human pose estimation has attracted enormous interest in the field of human action recognition. When the human pose is complex (such as pose distortion, pose reversal, etc.) or there is background interference (multi-target, shadow, etc.), the keypoints obtained by existing methods of human pose estimation often have incorrect positioning, category, and connection. This paper proposes a novel human pose estimation network KACNet via the keypoint association constraints. The Channel-1 of KACNet is constrained by the distance loss function to obtain the position of keypoints, and the Channel-2 of KACNet is constrained by the association loss function to obtain the relationship of keypoints. Then, the position and relationship of keypoints are fused by the weighted loss function to obtain the keypoints with accurate location, classification, and connection. Experiments on a large number of public datasets and Internet data show that our method can effectively suppress background interference to improve the accuracy of complex human pose estimation. Compared with state-of-the-art human pose estimation methods, the proposed methods can accurately locate, classify, and connect the human body keypoints robustly.

Highlights

Human pose estimation is one of the important branches in the field of computer vision and human action recognition [1]–[4]
Almost all human pose estimation methods based on deep learning have the following three characteristics: (i) The MS COCO [41] or MPII datasets are used as the training set, such as [9], [26], [37], [40]
Three loss functions are designed in KACNet: the distance loss function Ldis, association loss function Laso, and weighted loss function Lwei

Summary

INTRODUCTION

Human pose estimation is one of the important branches in the field of computer vision and human action recognition [1]–[4]. Almost all human pose estimation methods based on deep learning have the following three characteristics: (i) The MS COCO [41] or MPII datasets are used as the training set, such as [9], [26], [37], [40]. These two datasets mainly include regular upright pose data, such as playing football, playing badminton, walking, and so on. To improve the quality of complex human pose estimation, a novel keypoints association constraint network KACNet and evaluation index are proposed. Channel-1, and K feature maps F1 ∈ Rw×h×K are obtained after convolution operation; (iii) Send Ifm to the Channel-2, and K − 1 feature maps F2 ∈ Rw×h×(K−1) are obtained after convolution operation; (iv) Extract the keypoints in both F1 and F2, and compare it with the groundtruth keypoints to calculate the final keypoints

LOSS FUNCTION

NETWORK TRAINING

EVALUATION INDEX

EXPERIMENT

CONCLUSION