Dense Pose Estimation Research Articles

CoNPL: Consistency training framework with noise-aware pseudo labeling for dense pose estimation

Dense pose estimation faces hurdles due to the lack of costly precise pixel-level IUV labels. Existing methods aim to overcome it by regularizing model outputs or interpolating pseudo labels. However, conventional geometric transformations often fall short, and pseudo labels may introduce unwanted noise, leading to continued challenges in rectifying inaccurate estimations. We introduced a novel Consistency training framework with Noise-aware Pseudo Labeling (CoNPL) to tackle problems in learning from unlabeled pixels. CoNPL employs both weak and strong augmentations in a shared model to enhance robustness against aggressive transformations. To address noisy pseudo labels, CoNPL integrates a Noise-aware Pseudo Labeling (NPL) module, which consists of a Noise-Aware Module (NAM), and Noise-Resistant Learning (NRL) modules. NAM identifies misclassifications and incorrect UV coordinates using binary classification and regression, while NRL dynamically adjusts loss weights to filter out uncertain samples, thereby stabilizing learning from pseudo labels. Our method demonstrates a + 2.0% improvement in AP on the DensePose-COCO benchmark across different networks, achieving state-of-the-art performance. On the Ultrapose and DensePose-Chimps benchmark, our method also demonstrates a + 2.7% and + 3.0% improvement in AP.

SMPR: Single-stage multi-person pose regression

Existing multi-person pose estimators can be roughly divided into two-stage approaches (top-down and bottom-up approaches) and one-stage approaches. The two-stage methods either suffer high computational redundancy for additional person detectors or group keypoints heuristically after predicting all the instance-free keypoints. The recently proposed single-stage methods do not rely on the above two extra stages but have lower performance than the latest bottom-up approaches. In this work, a novel single-stage multi-person pose regression, termed SMPR, is presented. It follows the paradigm of dense prediction and predicts instance-aware keypoints from every location. Besides feature aggregation, we propose better strategies to define positive pose hypotheses for training which all play an important role in dense pose estimation. The network also learns the scores of estimated poses. The pose scoring strategy further improves the pose estimation performance by prioritizing superior poses during non-maximum suppression (NMS). We show that our method not only outperforms existing single-stage methods but also be competitive with the latest bottom-up methods, with 70.2 AP and 77.5 AP75 on the COCO test-dev pose benchmark. The code is available at https://github.com/cmdi-dlut/SMPR .

Single Person Dense Pose Estimation via Geometric Equivariance Consistency

We study the task of single person dense pose estimation. Specifically, given a human-centric image, we learn to map all human pixels onto a 3D, surface-based human body model. Existing methods approach this problem by fitting deep convolutional networks on sparse annotated points where the regression on both surface coordinate components for each body part is uncorrelated and optimized separately. In this work, we devise a novel, unified loss function that explicitly characterizes the correlation for surface coordinates regression, achieving significant improvements in both accuracy and efficiency. Furthermore, based on an observation that the image-to-surface correspondence is intrinsically invariant to geometric transformations from input images, we propose to enforce a geometric equivariance consistency on the target mapping, thereby allowing us to enable reliable supervision on large amounts of unlabeled pixels. We conduct comprehensive studies on the effectiveness of our approach using a quite simple network. Extensive experiments on the DensePose-COCO dataset show that our model achieves superior performance against previous state-of-the-art methods with much less computation complexity. We hope that our work would serve as a solid baseline for future study in the field. The code will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Johnqczhang/densepose.pytorch</uri> .

Densemask RCNN: A Hybrid Model for Skin Burn Image Classification and Severity Grading

In the medical image processing, automatic segmentation of burn images is one of the critical tasks in the classification of skin burn into normal and burn area. Traditional models identify the burns from the image and distinguish the region as burn and non-burn regions. However, the earlier models cannot accurately classify the wound region and also requires more time in the prediction of burns. Also, the burn depth analysis is an important factor for the calculation of the percentage of burn depth i.e. degree of severity is analyzed by Total body surface area (TBSA). For those issues, we design a hybrid approach named DenseMask Regional convolutional neural network (RCNN) approach for segmenting the skin burn region based on the various degrees of burn severity. In this, hybrid integration of Mask-region based convolution neural network CNN (Mask R-CNN) and dense pose estimation are integrated into DenseMask RCNN that calculate the full-body human pose and performs semantic segmentation. At first, we use the Residual Network with a dilated convolution using a weighted mapping model to generate the dense feature map. Then the feature map is fed into the Region proposal network (RPN) which utilizes a Feature pyramid network (FPN) to detect the objects at different scales of location and pyramid level from the input images. For the accurate alignment of pixel-to-pixel labels, we introduce a Region of interest (RoI)-pose align module that properly aligns the objects based on the human pose with the characteristics of scale, right-left, translation, and left–right flip to a standard scale. After the alignment task, a cascaded fully convolutional architecture is employed on the top of the RoI module that performs mask segmentation and dense pose regression task simultaneously. Finally, the transfer learning model classifies the detected burn regions into three classes of wound depths. Experimental analysis is performed on the burn dataset and the result obtained shows better accuracy than the state-of-art approaches.