PoiseNet: Dealing With Data Imbalance in DensePose

Abstract

Data imbalance, a foundational problem in machine learning, has received little attention in DensePose and has become one of the main obstacles in front of existing methods. We reveal two imbalances in DensePose: inter-surface imbalance and intra-surface imbalance. First, the human body parts can be of various sizes, making 3D surfaces contain different numbers of annotations. Classifiers trained on such unbalanced data suffer a decline on surfaces with fewer annotations. Second, annotations within each surface are not uniformly distributed. Regressors trained on such uneven data suffer a decline in 3D surface areas with sparse annotations. To solve these imbalances, we propose PoiseNet which integrates adaptive equalization loss (AEQL) and block balanced localization (BBL). Specifically, to address the inter-surface imbalance, AEQL adaptively reweights surfaces based on the number of annotations and the classification scores. BBL alleviates the intra-surface imbalance by unevenly blocking each surface according to annotation distribution. Experimental results on the DensePose-COCO dataset show that our PoiseNet surpasses baselines by up to 1.4 AP.