Multi-person Pose Estimation Research Articles

Monocular 3D Multi-Person Pose Estimation for On-Site Joint Flexion Assessment: A Case of Extreme Knee Flexion Detection.

Work-related musculoskeletal disorders (WMSDs) represent a significant health challenge for workers in construction environments, often arising from prolonged exposure to ergonomic risks associated with manual labor, awkward postures, and repetitive motions. These conditions not only lead to diminished worker productivity but also incur substantial economic costs for employers and healthcare systems alike. Thus, there is an urgent need for effective tools to assess and mitigate these ergonomic risks. This study proposes a novel monocular 3D multi-person pose estimation method designed to enhance ergonomic risk assessments in construction environments. Leveraging advanced computer vision and deep learning techniques, this approach accurately captures and analyzes the spatial dynamics of workers' postures, with a focus on detecting extreme knee flexion, a critical indicator of work-related musculoskeletal disorders (WMSDs). A pilot study conducted on an actual construction site demonstrated the method's feasibility and effectiveness, achieving an accurate detection rate for extreme flexion incidents that closely aligned with supervisory observations and worker self-reports. The proposed monocular approach enables universal applicability and enhances ergonomic analysis through 3D pose estimation and group pose recognition for timely interventions. Future efforts will focus on improving robustness and integration with health monitoring to reduce WMSDs and promote worker health.

Enhanced keypoint information and pose-weighted re-ID features for multi-person pose estimation and tracking

Enhanced keypoint information and pose-weighted re-ID features for multi-person pose estimation and tracking

MRSAPose: Multi-level routing sparse attention for multi-person pose estimation

Multi-person Pose Estimation (MPPE) aims to reconstruct human poses by locating and connecting keypoints of individuals in input images. The variability of human poses and the complexity of scenes make MPPE reliant on both local details and global structures, and the absence of either can lead to the generation of deformed poses. With the emergence of Transformer, the performance of MPPE has been significantly improved. However, due to self-attention computing attention scores between each pair of positions, the current Transformer-based MPPE exhibits high quadratic complexity. To address these issues, this paper proposes a novel pose estimation model, MRSAPose. MRSAPose utilizes Multi-level Routing Sparse Attention (MRSA) to dynamically select relevant regions for attention, reducing computational complexity and mitigating the impact of irrelevant regions. Furthermore, MRSAPose constructs a Transformer-CNN Parallel Interaction Block (T-CP block) through MRSA and Recursive Residual Gated Convolution (Res-gnConv), facilitating parallel learning of global and local information. By relying on multi-level routing algorithms and high-order spatial interactions conducted by recursive processing of adjacent features, T-CP block helps MRSAPose effectively alleviates the issues of occlusion and misalignment in pose estimation. On multiple challenging keypoint datasets, MRSAPose outperforms current state-of-the-art algorithms, particularly excelling in crowded and occluded scenes.

Overcoming Data Deficiency for Multi-Person Pose Estimation.

Building multi-person pose estimation (MPPE) models that can handle complex foreground and uncommon scenes is an important challenge in computer vision. Aside from designing novel models, strengthening training data is a promising direction but remains largely unexploited for the MPPE task. In this article, we systematically identify the key deficiencies of existing pose datasets that prevent the power of well-designed models from being fully exploited and propose the corresponding solutions. Specifically, we find that the traditional data augmentation techniques are inadequate in addressing the two key deficiencies, imbalanced instance complexity (IC) (evaluated by our new metric IC) and insufficient realistic scenes. To overcome these deficiencies, we propose a model-agnostic full-view data generation (Full-DG) method to enrich the training data from the perspectives of both poses and scenes. By hallucinating images with more balanced pose complexity and richer real-world scenes, Full-DG can help improve pose estimators' robustness and generalizability. In addition, we introduce a plug-and-play adaptive category-aware loss (AC-loss) to alleviate the severe pixel-level imbalance between keypoints and backgrounds (i.e., around 1:600). Full-DG together with AC-loss can be readily applied to both the bottom-up and top-down models to improve their accuracy. Notably, plugging into the representative estimators HigherHRNet and HRNet, our method achieves substantial performance gains of 1.0%-2.9% AP on the COCO benchmark, and 1.0%-5.1% AP on the CrowdPose benchmark.

Multi-Person Pose Regression With Distribution-Aware Single-Stage Models.

Understanding human posture is a challenging topic, which encompasses several tasks, e.g., pose estimation, body mesh recovery and pose tracking. In this article, we propose a novel Distribution-Aware Single-stage (DAS) model for the pose-related tasks. The proposed DAS model estimates human position and localizes joints simultaneously, which requires only a single pass. Meanwhile, we utilize normalizing flow to enable DAS to learn the true distribution of joint locations, rather than making simple Gaussian or Laplacian assumptions. This provides a pivotal prior and greatly boosts the accuracy of regression-based methods, thus making DAS achieve comparable performance to the volumetric-based methods. We also introduce a recursively update strategy to progressively approach the regression target, reducing the difficulty of regression and improving the regression performance. We further adapt DAS to multi-person mesh recovery and pose tracking tasks and achieve considerable performance on both tasks. Comprehensive experiments on CMU Panoptic and MuPoTS-3D demonstrate the superior efficiency of DAS, specifically 1.5 times speedup over previous best method, and its state-of-the-art accuracy for multi-person pose estimation. Extensive experiments on 3DPW and PoseTrack2018 indicate the effectiveness and efficiency of DAS for human body mesh recovery and pose tracking, respectively, which prove the generality of our proposed DAS model.

EFCPose: End-to-End Multi-Person Pose Estimation With Fully Convolutional Heads

EFCPose: End-to-End Multi-Person Pose Estimation With Fully Convolutional Heads

Real-time multi-camera 3D human pose estimation at the edge for industrial applications

There is an increasing interest in exploiting human pose estimation (HPE) software in human–machine interaction systems. Nevertheless, adopting such a computer vision application in real industrial scenarios is challenging. To overcome occlusion limitations, it requires multiple cameras, which in turn require multiple, distributed, and synchronized HPE software nodes running on resource-constrained edge devices. We address this challenge by presenting a real-time distributed 3D HPE platform, which consists of a set of 3D HPE software nodes on edge devices (i.e., one per camera) to redundantly extrapolate the human pose from different points of view. A centralized aggregator collects the pose information through a shared communication network and merges them, in real time, through a pipeline of filtering, clustering and association algorithms. It addresses network communication issues (e.g., delay and bandwidth variability) through a two-levels synchronization, and supports both single and multi-person pose estimation. We present the evaluation results with a real case of study (i.e., HPE for human–machine interaction in an intelligent manufacturing line), in which the platform accuracy and scalability are compared with state-of-the-art approaches and with a marker-based infra-red motion capture system.

DRSI-Net: Dual-residual spatial interaction network for multi-person pose estimation

Multi-person pose estimation (MPPE), which aims to locate the key points for all persons in the frames, is an active research branch of computer vision. Variable human poses and complex scenes make MPPE dependent on local details and global structures; their absence may cause key point feature misalignment. In this case, high-order spatial interactions that can effectively link the local and global information of features are particularly important. However, most methods do not include spatial interactions. A few methods have low-order spatial interactions, but achieving a good balance between accuracy and complexity is challenging. To address the above problems, a dual-residual spatial interaction network (DRSI-Net) for MPPE with high accuracy and low complexity is proposed herein. Compared to other methods, DRSI-Net recursively performs residual spatial information interactions on the neighbouring features so that more useful spatial information can be retained and more similarities can be obtained between shallow and deep extracted features. The channel and spatial dual attention mechanism introduced in the multi-scale feature fusion also helps the network to adaptively focus on features relevant to the target key points and further refine the generated poses. Simultaneously, by optimising the interactive channel dimensions and dividing the gradient flow, the spatial interaction module is designed to be lightweight, thus reducing the complexity of the network. According to the experimental results on the COCO dataset, the proposed DRSI-Net outperforms other state-of-the-art methods in accuracy and complexity.

Review on Human Pose Estimation using AI Fitness Tracker

Human pose estimation, a fundamental computer vision task, has witnessed significant advancements with the advent of deep learning techniques. This paper provides an overview of recent developments in human pose estimation, focusing on the application of convolutional neural networks (CNNs) and recurrent neural networks (RNNs). We explore the distinctions between 2D and 3D pose estimation, highlighting the challenges and solutions for each. The paper discusses the various techniques for single-person and multi-person pose estimation, as well as the trade-offs between part-based and holistic approaches. Additionally, we delve into the real-time and offline aspects of pose estimation and the significance of temporal analysis in videos. Lastly, we touch on the translation of 2D pose information into 3D space. This review underscores the practical implications of human pose estimation in diverse fields such as sports analysis, healthcare, surveillance, and entertainment. The dynamic nature of this field, fueled by ongoing research and technological advancements, promises to unlock new possibilities for human-computer interaction, augmented reality, and beyond. This abstract provides an overview of the paper's focus on deep learning in the context of human pose estimation and highlights the key aspects of the field. You can further tailor it to your specific paper or presentation needs. Keywords: Human pose estimation, camera, Artificial intelligence, Machine learning

YOLO-Rlepose: Improved YOLO Based on Swin Transformer and Rle-Oks Loss for Multi-Person Pose Estimation

In recent years, there has been significant progress in human pose estimation, fueled by the widespread adoption of deep convolutional neural networks. However, despite these advancements, multi-person 2D pose estimation still remains highly challenging due to factors such as occlusion, noise, and non-rigid body movements. Currently, most multi-person pose estimation approaches handle joint localization and association separately. This study proposes a direct regression-based method to estimate the 2D human pose from a single image. The authors name this network YOLO-Rlepose. Compared to traditional methods, YOLO-Rlepose leverages Transformer models to better capture global dependencies between image feature blocks and preserves sufficient spatial information for keypoint detection through a multi-head self-attention mechanism. To further improve the accuracy of the YOLO-Rlepose model, this paper proposes the following enhancements. Firstly, this study introduces the C3 Module with Swin Transformer (C3STR). This module builds upon the C3 module in You Only Look Once (YOLO) by incorporating a Swin Transformer branch, enhancing the YOLO-Rlepose model’s ability to capture global information and rich contextual information. Next, a novel loss function named Rle-Oks loss is proposed. The loss function facilitates the training process by learning the distributional changes through Residual Log-likelihood Estimation. To assign different weights based on the importance of different keypoints in the human body, this study introduces a weight coefficient into the loss function. The experiments proved the efficiency of the proposed YOLO-Rlepose model. On the COCO dataset, the model outperforms the previous SOTA method by 2.11% in AP.

CONet: Crowd and occlusion-aware network for occluded human pose estimation

Human pose estimation has numerous applications in motion recognition, virtual reality, human–computer interaction, and other related fields. However, multi-person pose estimation in crowded and occluded scenes is challenging. One major issue about the current top-down human pose estimation approaches is that they are limited to predicting the pose of a single person, even when the bounding box contains multiple individuals. To address this problem, we propose a novel Crowd and Occlusion-aware Network (CONet) using a divide-and-conquer strategy. Our approach includes a Crowd and Occlusion-aware Head (COHead) which estimates the pose of both the occluder and the occluded person using two separate branches. We also use the attention mechanism to guide the branches for differentiated learning, aiming to improve feature representation. Additionally, we propose a novel interference point loss to enhance the model’s anti-interference ability. Our CONet is simple yet effective, and it outperforms the state-of-the-art model by +1.6 AP, achieving 71.6 AP on CrowdPose. Our proposed model has achieved state-of-the-art results on the CrowdPose dataset, demonstrating its effectiveness in improving the accuracy of human pose estimation in crowded and occluded scenes. This achievement highlights the potential of our model in many real-world applications where accurate human pose estimation is crucial, such as surveillance, sports analysis, and human–computer interaction.

Rethinking the Person Localization for Single-Stage Multi-Person Pose Estimation

Single-stage models for multi-person pose estimation have garnered significant attention due to their streamlined approach in generating person position localization and body structure perception in a single pass. These two parts, however, are processed individually by existing methods, leading to suboptimal results, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , candidates with high confidences for person localization while poor structure estimations. To this end, we propose a simple yet effective approach, namely Structure-guided Person Localization (SPL), jointly leveraging the advantages of the two aspects to solve the multi-person pose estimation problem, with two complementary novelties. First, we propose to incorporate body structure perception to guide person position localization, consequently, we introduce the Structure-guided Center Learning (SCL) to unify the quality of the body structure perception in the displacement map with the confidence of the person existence in the center map, thus achieving more accurate keypoint position localization results even with extreme poses. Second, to facilitate the end-to-end training of SPL, we propose the efficient Agency-based Scale-adaptive Learning (ASL). Specifically, we predict an agency map of the same size as the center map, which focuses on the foreground area and can adaptively adjust the scale size for each central area with the body structure perception confidence. Comprehensive experiments on challenging benchmarks including COCO and CrowdPose clearly verify the superiority of our framework, which achieves new state-of-the-art single-stage multi-person pose estimation results. Specifically, SPL obtains 72.1 AP scores and 69.5 AP scores in COCO test-dev2017 and CrowdPose test set, respectively.

Lightweight Multiperson Pose Estimation With Staggered Alignment Self-Distillation

Lightweight Multiperson Pose Estimation With Staggered Alignment Self-Distillation

Multi-Person Pose Tracking With Sparse Key-Point Flow Estimation and Hierarchical Graph Distance Minimization.

In this paper, we propose a novel framework for multi-person pose estimation and tracking on challenging scenarios. In view of occlusions and motion blurs which hinder the performance of pose tracking, we proposed to model humans as graphs and perform pose estimation and tracking by concentrating on the visible parts of human bodies which are informative about complete skeletons under incomplete observations. Specifically, the proposed framework involves three parts: (i) A Sparse Key-point Flow Estimating Module (SKFEM) and a Hierarchical Graph Distance Minimizing Module (HGMM) for estimating pixel-level and human-level motion, respectively; (ii) Pixel-level appearance consistency and human-level structural consistency are combined in measuring the visibility scores of body joints. The scores guide the pose estimator to predict complete skeletons by observing high-visibility parts, under the assumption that visible and invisible parts are inherently correlated in human part graphs. The pose estimator is iteratively fine-tuned to achieve this capability; (iii) Multiple historical frames are combined to benefit tracking which is implemented using HGMM. The proposed approach not only achieves state-of-the-art performance on PoseTrack datasets but also contributes to significant improvements in other tasks such as human-related anomaly detection.

Learning Visual Clue for UWB-based multi-person pose estimation

Compared to camera image-based methods, radio frequency (RF) based pose estimation has great potential for use in situations where the field of view is obstructed. In this paper, we present a novel RF-based Pose Estimation framework with Transformer (RPET) that operates in a fully end-to-end fashion and uses an easy-to-install portable radar. RPET eliminates the need for complex preprocessing and hand-crafted post-processing modules, such as region-of-interest (RoI) cropping, non-maximum suppression (NMS), and keypoint grouping. We also introduce a novel concept called Visual Clue (VC), which mimics a pose feature represented in image-based methods and improves the learning performance of multi-person pose estimation from RF signals. Our experimental results demonstrate the effectiveness of VC and the generalizability of our model to different environmental conditions, including changes in location and obstructed views.

A Systematic Review of Recent Deep Learning Approaches for 3D Human Pose Estimation.

Three-dimensional human pose estimation has made significant advancements through the integration of deep learning techniques. This survey provides a comprehensive review of recent 3D human pose estimation methods, with a focus on monocular images, videos, and multi-view cameras. Our approach stands out through a systematic literature review methodology, ensuring an up-to-date and meticulous overview. Unlike many existing surveys that categorize approaches based on learning paradigms, our survey offers a fresh perspective, delving deeper into the subject. For image-based approaches, we not only follow existing categorizations but also introduce and compare significant 2D models. Additionally, we provide a comparative analysis of these methods, enhancing the understanding of image-based pose estimation techniques. In the realm of video-based approaches, we categorize them based on the types of models used to capture inter-frame information. Furthermore, in the context of multi-person pose estimation, our survey uniquely differentiates between approaches focusing on relative poses and those addressing absolute poses. Our survey aims to serve as a pivotal resource for researchers, highlighting state-of-the-art deep learning strategies and identifying promising directions for future exploration in 3D human pose estimation.

FDNet: Feature decoupling for single-stage pose estimation in complex scenes

Severe error accumulation is a key reason that makes existing multi-person pose estimation challenging in complex scenes. Single-stage pose estimation methods detect human and keypoints in parallel, avoiding serial error accumulation compared to two-stage methods. However, two distinct tasks share identical features in the existing method, which causes error accumulation. Therefore, we propose a Feature Decoupling Network (FDNet) to further extend the pipeline of single-stage methods and reconstruct task-specific features, including Feature Decoupling Module (FDM), Human Aware Loss (HAL) and Keypoint Aware Loss (KAL). FDM can adaptively perceive spatial and channel features and allocate separate feature domains for various tasks. To learn distinctive feature representation for compact human bodies, HAL and KAL measure and suppress feature similarities and distances for different human and keypoints, thus alienating feature interference. Experiments on crowded datasets show that our method is superior, outperforming the state-of-the-art method CID by 1.5%, 0.8% on OCHuman and CrowdPose.

The comparison of top-down and bottom-up methods in multi-person pose estimation

2D multi-person pose estimation is a process to detect all bodies in a two-dimensional picture. The main purpose of this report is to discuss the difference between top-down and bottom- up method in multi-person human pose estimation. This estimation will focus on many people in one picture. There are two popular methods in this area. The first is top-down method, which is to find people firstly and then to detect the body of each people. On the other hand, the bottom-up method is to detect body parts and then find each person. Their comparison and analyse in algorithm, speed and accuracy may help researchers to find more suitable methods when they research in human pose estimation. Generally, top-down methods have higher accuracy than bottom-up methods but have higher speed.

Enhanced 3D Pose Estimation in Multi-Person, Multi-View Scenarios through Unsupervised Domain Adaptation with Dropout Discriminator.

Data-driven pose estimation methods often assume equal distributions between training and test data. However, in reality, this assumption does not always hold true, leading to significant performance degradation due to distribution mismatches. In this study, our objective is to enhance the cross-domain robustness of multi-view, multi-person 3D pose estimation. We tackle the domain shift challenge through three key approaches: (1) A domain adaptation component is introduced to improve estimation accuracy for specific target domains. (2) By incorporating a dropout mechanism, we train a more reliable model tailored to the target domain. (3) Transferable Parameter Learning is employed to retain crucial parameters for learning domain-invariant data. The foundation for these approaches lies in the H-divergence theory and the lottery ticket hypothesis, which are realized through adversarial training by learning domain classifiers. Our proposed methodology is evaluated using three datasets: Panoptic, Shelf, and Campus, allowing us to assess its efficacy in addressing domain shifts in multi-view, multi-person pose estimation. Both qualitative and quantitative experiments demonstrate that our algorithm performs well in two different domain shift scenarios.

Towards reliable multi-person pose estimation using Conditional Random Fields

Multi-person pose estimation is the task of estimating the coordinates of body joints and predicting the body poses for multiple people in an images. This problem has made breakthroughs in recent years, but the solutions still suffer from some shortcomings. A serious weakness of the state-of-the-art models is the number of poses detected by these models which is generally much larger than the actual number of human instances in the input image. This makes the existing models unreliable and thus unusable in real-world tasks. In this paper, we propose a more reliable multi-person pose estimation method consisting of three main blocks: a top-down multi-person pose estimation, a human detection, and a pose selection block. The proposed method incorporates the bounding box of the segmented objects to select the best subset of the initial pose set. We formulate the pose selection problem using Conditional Random Fields. First, we introduce a set of potential functions to form a general probability model. Then an inference algorithm is proposed to select the best poses which maximize the probability function. Finally, the proposed solution is implemented by a neural network. The proposed pose selection model is a model-agnostic method that can be easily used in conjunction with other pose estimation and object detection models. Experiments demonstrate that the reliability and precision of the proposed model are higher than those of the state-of-the-art models.