Single-person Pose Estimation Research Articles

A Compact and Powerful Single-Stage Network for Multi-Person Pose Estimation

Multi-person pose estimation generally follows top-down and bottom-up paradigms. The top-down paradigm detects all human boxes and then performs single-person pose estimation on each ROI. The bottom-up paradigm locates identity-free keypoints and then groups them into individuals. Both of them use an extra stage to build the relationship between human instance and corresponding keypoints (e.g., human detection in a top-down manner or a grouping process in a bottom-up manner). The extra stage leads to a high computation cost and a redundant two-stage pipeline. To address the above issue, we introduce a fine-grained body representation method. Concretely, the human body is divided into several local parts and each part is represented by an adaptive point. The novel body representation is able to sufficiently encode the diverse pose information and effectively model the relationship between human instance and corresponding keypoints in a single-forward pass. With the proposed body representation, we further introduce a compact single-stage multi-person pose regression network, called AdaptivePose++, which is the extended version of AAAI-22 paper AdaptivePose. During inference, our proposed network only needs a single-step decode operation to estimate the multi-person pose without complex post-processes and refinements. Without any bells and whistles, we achieve the most competitive performance on representative 2D pose estimation benchmarks MS COCO and CrowdPose in terms of accuracy and speed. In particular, AdaptivePose++ outperforms the state-of-the-art SWAHR-W48 and CenterGroup-W48 by 3.2 AP and 1.4 AP on COCO mini-val with faster inference speed. Furthermore, the outstanding performance on 3D pose estimation datasets MuCo-3DHP and MuPoTS-3D further demonstrates its effectiveness and generalizability on 3D scenes.

Improvement of Human Pose Estimation and Processing With the Intensive Feature Consistency Network

The modeling of human body kye-points is the most significant aspect of pose estimation appropriately. Computer vision algorithm identifies human pose, body-movement, and action in many ways. Most of the previous works taken advantage for finding accuracy or efficiency in terms of speed. However, many techniques suffer for intensive computational demands with low-latency or higher proceeding speed. We have designed a unique approach for single-person pose estimation and action recognition which is well suited for fitness application and mobility activities. The proposed framework has been developed with a base network that provides an initial pose to further refinement through Intensive Feature Consistency (IFC) network. The IFC network enforces high-level constraints on the global body intensity correction and local body part adjustments. The proposed module reduces the impact of body joint movement diversity by interpreting long-term consistent view. We have illustrated the effectiveness of proposed framework through pose estimation accuracy improvement with two benchmark datasets. Which is specified state-of the- art performance of IFC network under the required real-time processing speed on the CPU platform. The IFC network has improved 99.1% of PCK body and 94.7% of PCK torso accuracy under 31 FPS, which is comparatively higher than the existing work.

Knowledge Distillation for Lightweight 2D Single-Person Pose Estimation

The current state-of-the-art single-person pose estimation methods require heavily parameterized models for accurate predictions. A promising technique to achieve accurate yet lightweight pose estimation is knowledge distillation. However, existing pose knowledge distillation methods rely on the most common large basic building blocks and a complex multi-branch architecture. In this study, we propose a Single-branch Lightweight Knowledge Distillation method to increase pose distillation efficiency for 2D Single-person pose estimation, termed SLKD2S. First, we design a novel single-branch pose knowledge distillation framework, which is composed of connected lightweight pose estimation stages. Second, we utilize a special pose distillation loss based on the joint confidence map. Finally, we only keep the initial stage and the first refinement stage to achieve a good performance. Extensive experiments on two standard benchmark datasets show the superiority of the proposed SLKD2S in terms of cost and accuracy, and the average detection accuracies are increased by 1.43% and 2.74% compared with the top-performing pose distillation method, respectively.

A Lightweight Top-Down Multi-Person Pose Estimation Method Based on Symmetric Transformation and Global Matching

The top-down human pose estimation method usually faces the following problems: (i) The target detection result is not well applied in the pose estimation network. (ii) Difficulty of human detection in the crowded state. (iii) The complicated model leads to a long training time. Aiming at the issues above, a lightweight multi-person pose estimation method based on symmetric transformation and global matching is proposed. Symmetric transformation module adds spatial transformation network(STN) and spatial de-transformer network(SDTN) before and after the single-person pose estimation(SPPE) to extract high-quality single-person pose regions from inaccurate human candidate frames. Global matching method is used to transform the key point prediction problem into the optimal matching problem of the human body-key point graph, and solve the questions of misjudgment and error detection of pose estimation in the crowded state. Finally, depth wise separable convolution and inverted residual model are used to reduce the complexity of model, so as to improve the running speed while balancing the accuracy of the algorithm. Experiments show that the algorithm proposed in this paper not only enhance the overall performance of the multi-person posture estimation network in the crowded state, but also improves the running speed significantly, which further confirms the effectiveness and competitiveness of this algorithm.

Monocular 3D multi-person pose estimation via predicting factorized correction factors

Despite the great achievement of 3D human pose estimation, recovering the 3D poses of multiple persons in a single image is still a challenging problem. In this paper, we focus on one specific problem in 3D multi-person pose estimation (3D-MPPE): estimating the absolute 3D human poses. We proposed a pipeline consists of human detection, absolute 3D human root localization, and root-relative 3D single-person pose estimation modules. For the absolute 3D human root localization task, we propose a decoupling dual-branch structure to reconstruct the height of the human body, and further output the depth and localization of the 3D human root in the camera coordinate system. Furthermore, a data augmentation strategy is presented to tackle occlusions, such that our model can effectively estimate the root localization with the incomplete bounding boxes. For the 3D human relative pose estimation task, we use the attention mechanism to capture the correlation between human joint coordinates and further improve the accuracy of relative pose estimation. Finally, we merge the absolute depth of human and the relative 3D pose to output the absolute 3D human pose.

AnimePose: Multi-person 3D pose estimation and animation

3D animation of human body movement is quite challenging as it involves using a huge setup with several motion trackers all over the persons body to track the movements of every limb. This is time- consuming and may cause the person discomfort in wearing exoskeleton body suits with motion sensors. In this work, we present a trivial yet effective solution to generate simple 3D animation of human movement of multiple persons from a 2D video using deep learning. Although significant improvement has been achieved recently in 3D human pose estimation, most of the prior works work well in case of single person pose estimation and multi-person pose estimation is still a challenging problem. In this work, we firstly propose a supervised multi-person 3D pose estimation and animation framework namely AnimePose for a given input RGB video sequence. The pipeline of the proposed system consists of various modules: i) Multi-Person 2D pose estimation, ii) Depth Map estimation, iii) Lifting 2D poses to 3D poses, iv) Person trajectory prediction and human pose tracking. Our proposed system produces comparable results on previous state-of-the-art 3D multi-person pose estimation methods on publicly available dataset MuPoTS-3D dataset and it also outperforms previous competing human pose tracking methods by a significant margin of 11.7% performance gain on MOTA score on Posetrack 2018 dataset.

Hope: heatmap and offset for pose estimation

The progress on human pose estimation by deep neural networks has been significantly advanced in recent years. However, the problem of precision loss caused by the prediction of the coordinates back to the original image has been neglected. In this paper, we propose a simple but effective method using Heatmap and Offset for Pose Estimation (HOPE). In order to solve the human pose estimation problem, firstly a general top-down method is used in HOPE to generate the human detection box based on a detector, and then the keypoints in each cropped box image are located. To alleviate the precision loss of mapping process, HOPE embeds the coordinate offset into the structure of the neural network, allowing the network to self-learn the slight offset in the mapping process in an end-to-end manner, which improves the accuracy in the current field of pose estimation. Experimental results on the multi-person pose estimation dataset MSCOCO, the single-person pose estimation dataset MPII and CrowdPose Pose Estimation dataset indicate that our method achieves state-of-the-art performance in terms of accuracy and computational complexity.

A survey on Pose Estimation using Deep Convolutional Neural Networks

Human pose estimation is a technique, which identifies the human body’s landmarks in images and videos. Human pose estimation can be divided into single person pose estimation and multi-person pose estimation, also an estimated human poses in crowded places as well as in videos. Depends upon the application such as activity recognition, Animation, Sports, Augmented reality, etc., Pose estimation output can be in 2D or 3D coordinate format. 3D pose estimation is estimated considering joint angles in 2D. Some challenges like small and barely visible joints, strong articulations, occlusions, clothing, and lighting changes increase difficulty in estimating pose. Remarkable progress has gained in the field of human pose estimation using Deep learning-based CNN models. In this paper, we compare and summarized various deep learning models for pose estimation of a single person and multi-person.

EfficientPose: Scalable single-person pose estimation

Single-person human pose estimation facilitates markerless movement analysis in sports, as well as in clinical applications. Still, state-of-the-art models for human pose estimation generally do not meet the requirements of real-life applications. The proliferation of deep learning techniques has resulted in the development of many advanced approaches. However, with the progresses in the field, more complex and inefficient models have also been introduced, which have caused tremendous increases in computational demands. To cope with these complexity and inefficiency challenges, we propose a novel convolutional neural network architecture, called EfficientPose, which exploits recently proposed EfficientNets in order to deliver efficient and scalable single-person pose estimation. EfficientPose is a family of models harnessing an effective multi-scale feature extractor and computationally efficient detection blocks using mobile inverted bottleneck convolutions, while at the same time ensuring that the precision of the pose configurations is still improved. Due to its low complexity and efficiency, EfficientPose enables real-world applications on edge devices by limiting the memory footprint and computational cost. The results from our experiments, using the challenging MPII single-person benchmark, show that the proposed EfficientPose models substantially outperform the widely-used OpenPose model both in terms of accuracy and computational efficiency. In particular, our top-performing model achieves state-of-the-art accuracy on single-person MPII, with low-complexity ConvNets.

Improving Multiperson Pose Estimation by Mask-aware Deep Reinforcement Learning

Research on single-person pose estimation based on deep neural networks has recently witnessed progress in both accuracy and execution efficiency. However, multiperson pose estimation is still a challenging topic, partially because the object regions are selected greedily from proposals via class-agnostic nonmaximum suppression (NMS), and the misalignment in the redundant detection yields inaccurate human poses. Therefore, we consider how to obtain the optimal input in human pose estimation under conditions in which intermediate label information is not available. As supervised learning–based alignment does not generalize well to unseen samples in the human pose space, in this article, we present a mask-aware deep reinforcement learning approach to modify the detection result. We use mask information to remove the adverse effects from the cluttered background and to select the optimal action according to the revised reward function. We also propose a new regularization term to punish joints that are outside of the silhouette region in the human pose estimation stage. We evaluate our approach on the MPII Multiperson dataset and the MS-COCO Keypoints Challenge. The results show that our approach yields competing inference results when it is compared to the other state-of-the-art approaches.

Hybrid Refinement-Correction Heatmaps for Human Pose Estimation

In this paper, we present a method (Hybrid-Pose) to improve human pose estimation in images. We adopt Stacked Hourglass Networks to design two convolutional neural network models, RNet for pose refinement and CNet for pose correction. The CNet (Correction Network) guides the pose refinement RNet (Refinement Network) to correct the joint location before generating the final pose. Each of the two models is composed of four hourglasses, and each hourglass generates a group of detection heatmaps for the joints. The RNet model hourglasses have the same structure. However, the CNet model is designed with hourglasses of different structures for pose guidance. Since the pose estimation in RGB images is very sensitive to the image scene, our proposed approach generates multiple outputs of detection heatmaps to broaden the searching scope for the correct joints locations. We use the RNet model to refine the joints locations in each hourglass stage horizontally, then the heatmaps of each stage are fused with the heatmaps of all the CNet model hourglasses vertically in a hybrid manner. Our method shows competitive results with the existing state-of-the-art approaches on MPII and FLIC benchmark datasets. Although our proposed method focuses on improving single-person pose estimation, we also show the influence of this improvement on multi-person pose estimation by detecting multiple people using SSD detector, then estimating the pose of each person individually.

A Fast Teacher Pose Estimation Framework Base on Kernelized Correlation Filter and Spatial Transformed High-Resolution Network

Abstract Smart Education gradually becomes more of a concern for people, it providing sharable education resources, diversified teaching methods and data-based teaching evaluation methods. Virtual teacher is a part of smart education and it helps on improving learners’ cognitive level. The difficulty of virtual teacher technology lies in accurately recording and reproducing the actions and postures of teachers. It requires an accurate and fast teacher pose estimation algorithm as a support. This paper proposes a teacher pose estimation framework based on KCF and ST-HRN. Firstly, the target detection network YOLOv3 and the similarity algorithm based on SIFT feature are used to predict the position of the teacher in the video, and the KCF target tracking algorithm is used to accelerate the calculation. Secondly, based on the high-resolution network, the ST-HRN single-person pose estimation model is proposed and trained, and the average PCKH@0.5=90.2% accuracy is obtained on the MPII dataset. Finally, the framework is tested on class teaching videos and have 30.8% estimate speed promotion by implementing tracking algorithm.

Multi-person pose estimation based on a deep convolutional neural network

Human motion recognition based on computer vision plays an important role in many fields, such as video surveillance, virtual reality, and medical care. To solve the inaccurate multi-person pose estimation problem and improve the generalizability of the extracted features, this paper proposes a multi-person pose estimation method based on a deep convolutional neural network. This method mainly relies on a top-down structure which includes two stages. In the first stage, the bounding boxes that are likely to contain people are first detected by an improved faster R-CNN. Individuals in the complex scenario are then tailored by box cropping. In the second stage, we combine heatmap detection with coordinate regression to address the single person pose estimation problem. Specially, a deep convolutional ResNet is employed to produce heatmaps of human body. The precise location of each joint is achieved by the fully connected conditional random field. Experimental results demonstrate our method achieves comparable performance with the state-of-the-art ones.