Pose Estimation Method Research Articles

Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video.

Markerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the more promising single-view technology has defects such as low accuracy and reliability, which seriously limits clinical application. This study proposes a high-resolution graph convolutional multilayer perception (HGcnMLP) human 3D pose estimation framework for smartphone monocular videos and estimates 15 healthy adults and 12 patients with musculoskeletal disorders (sarcopenia and osteoarthritis) gait spatiotemporal, knee angle, and center-of-mass (COM) velocity parameters, etc., and compared with the VICON gold standard system. The results show that most of the calculated parameters have excellent reliability (VICON, ICC (2, k): 0.853-0.982; Phone, ICC (2, k): 0.839-0.975) and validity (Pearson r: 0.808-0.978, p0.05). In addition, the proposed system can better evaluate human gait balance ability, and the K-means++ clustering algorithm can successfully distinguish patients into different recovery level groups. This study verifies the potential of a single smartphone video for 3D human pose estimation for rehabilitation auxiliary diagnosis and balance level recognition, and is an effective attempt at the clinical application of emerging computer vision technology. In the future, it is hoped that the corresponding smartphone program will be developed to provide a low-cost, effective, and simple new tool for remote monitoring and rehabilitation assessment of patients.

Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet).

As an important direction in computer vision, human pose estimation has received extensive attention in recent years. A High-Resolution Network (HRNet) can achieve effective estimation results as a classical human pose estimation method. However, the complex structure of the model is not conducive to deployment under limited computer resources. Therefore, an improved Efficient and Lightweight HRNet (EL-HRNet) model is proposed. In detail, point-wise and grouped convolutions were used to construct a lightweight residual module, replacing the original 3 × 3 module to reduce the parameters. To compensate for the information loss caused by the network's lightweight nature, the Convolutional Block Attention Module (CBAM) is introduced after the new lightweight residual module to construct the Lightweight Attention Basicblock (LA-Basicblock) module to achieve high-precision human pose estimation. To verify the effectiveness of the proposed EL-HRNet, experiments were carried out using the COCO2017 and MPII datasets. The experimental results show that the EL-HRNet model requires only 5 million parameters and 2.0 GFlops calculations and achieves an AP score of 67.1% on the COCO2017 validation set. In addition, PCKh@0.5mean is 87.7% on the MPII validation set, and EL-HRNet shows a good balance between model complexity and human pose estimation accuracy.

Fishing risky behavior recognition based on adaptive transformer, reinforcement learning and stochastic configuration networks

The fishing behavior in a high-voltage environment may lead to electric shock accidents. This paper proposes a fishing risky behavior recognition method based on adaptive Transformer, reinforcement learning and stochastic configuration networks (SCNs). Firstly, the fishing behavior image sample training set is evaluated using the actor network based on linear entropy in the reinforcement learning module to select suitable training samples that possess abundant straight-line elements. Subsequently, the selected training samples are fed into a multi-scale spatial deep convolution to extract continuous spatial features of elongated objects. The deformable Transformer network, enhanced with adaptive encoding and decoding layers through SCNs, is used to obtain the position and detection results of the fishing rod. The trained Transformer model is evaluated by a defined credibility evaluation metric for acquiring rewards to update the training sample set iteratively. Then, an adaptive adjustment mechanism is constructed for the encoding and decoding layers of the adaptive Transformer network to establish a library of adaptive Transformer models with different encoding and decoding layer levels to accommodate the feature requirements of training samples in various scenarios. Finally, the blending learning method for combining the detection results from the model library is integrated with human body object detection and pose estimation methods, and a predefined expert system is utilized for logic reasoning on fishing risky behavior. Experimental results demonstrate the effectiveness of the proposed method in this paper.

Interactive Game Development and Evaluation Based on Device-Free Pose Estimation Techniques

In recent years, significant advancements have been made in pose estimation methods. These methods can be broadly divided into two categories: device-based and device-free. Device-based methods, such as virtual reality data gloves and marker-based motion capture, are known for their accuracy. However, these require specific equipment, making them less accessible for the public. On the other hand, device-free methods need no device to recognize human movement. They usually use cameras and estimation algorithms to recognize the body parts. Owing to evolving artificial intelligence (AI) technology, estimation accuracy has increased and we adopt one of the device-free methods to develop two interactive games, “Brain Wall” and “Touch de Pose”. “Brain Wall” challenges players to imitate a silhouette, scoring them based on the accuracy of their pose compared to the silhouette using the Intersection over Union metric. The game encourages competitiveness and participation through a leader board system. “Touch de Pose” allows players to choose a pose theme and the players are required to position specific body parts within the displayed circles on the screen. “Touch de Pose” also includes real-time evaluation of the player’s pose and combines their image with themed background images generated by generative AI, some of which demonstrate the limitations of the technology. The games were showcased at a university open campus event, receiving overwhelmingly positive feedback: 95.6% satisfaction for “Brain Wall” and 98.1% for “Touch de Pose”. Our study shows that pose estimation-based interactive games show immense potential in generating interest in technology for those who are unfamiliar with information and communication technology. Additionally, the full-body engagement aspect of these games could also play a role in promoting physical activity as a regular habit.

Multi-Person 3D Pose Estimation With Occlusion Reasoning

The performance of existing methods for multi-person 3D pose estimation in crowded scenes is still limited, due to the challenge of heavy overlapping among persons. Attempt to address this issue, we propose a progressive inference scheme, i.e., Articulation-aware Knowledge Exploration (AKE), to improve the multi-person 3D pose models on those samples with complex occlusions at the inference stage. We argue it is beneficial to explore the underlying articulated information/knowledge of the human body, which helps to further correct the predicted poses in those samples. To exploit such information, we propose an iterative scheme to achieve a self-improving loop for keypoint association. Specifically, we introduce a kinematic validation module for locating unreasonable articulations and an occluded-keypoint discovering module for discovering occluded articulations. Extensive experiments on two challenging benchmarks under both weakly-supervised and fully-supervised settings demonstrate the superiority and generalization ability of our proposed method for crowded scenes.

Enhanced Monocular Visual Odometry: A Robust Appearance-Based Method for Accurate Vehicle Pose Estimation

Enhanced Monocular Visual Odometry: A Robust Appearance-Based Method for Accurate Vehicle Pose Estimation

Advancing 6-DoF Instrument Pose Estimation in Variable X-Ray Imaging Geometries.

Accurate 6-DoF pose estimation of surgical instruments during minimally invasive surgeries can substantially improve treatment strategies and eventual surgical outcome. Existing deep learning methods have achieved accurate results, but they require custom approaches for each object and laborious setup and training environments often stretching to extensive simulations, whilst lacking real-time computation. We propose a general-purpose approach of data acquisition for 6-DoF pose estimation tasks in X-ray systems, a novel and general purpose YOLOv5-6D pose architecture for accurate and fast object pose estimation and a complete method for surgical screw pose estimation under acquisition geometry consideration from a monocular cone-beam X-ray image. The proposed YOLOv5-6D pose model achieves competitive results on public benchmarks whilst being considerably faster at 42 FPS on GPU. In addition, the method generalizes across varying X-ray acquisition geometry and semantic image complexity to enable accurate pose estimation over different domains. Finally, the proposed approach is tested for bone-screw pose estimation for computer-aided guidance during spine surgeries. The model achieves a 92.41% by the 0.1·d ADD-S metric, demonstrating a promising approach for enhancing surgical precision and patient outcomes. The code for YOLOv5-6D is publicly available at https://github.com/cviviers/YOLOv5-6D-Pose.

Line-Based 6-DoF Object Pose Estimation and Tracking With an Event Camera.

Pose estimation and tracking of objects is a fundamental application in 3D vision. Event cameras possess remarkable attributes such as high dynamic range, low latency, and resilience against motion blur, which enables them to address challenging high dynamic range scenes or high-speed motion. These features make event cameras an ideal complement over standard cameras for object pose estimation. In this work, we propose a line-based robust pose estimation and tracking method for planar or non-planar objects using an event camera. Firstly, we extract object lines directly from events, then provide an initial pose using a globally-optimal Branch-and-Bound approach, where 2D-3D line correspondences are not known in advance. Subsequently, we utilize event-line matching to establish correspondences between 2D events and 3D models. Furthermore, object poses are refined and continuously tracked by minimizing event-line distances. Events are assigned different weights based on these distances, employing robust estimation algorithms. To evaluate the precision of the proposed methods in object pose estimation and tracking, we have devised and established an event-based moving object dataset. Compared against state-of-the-art methods, the robustness and accuracy of our methods have been validated both on synthetic experiments and the proposed dataset. The source code is available at https://github.com/Zibin6/LOPET.

Recognition and position estimation method for stacked untextured parts

To address the challenge of recognizing and estimating the position of untextured stacked parts, which are common in industrial environments, this study proposes an integrated approach that incorporates the YOLOv7 target detection algorithm and point cloud alignment techniques. First, the YOLOv7 algorithm is utilized to quickly identify and locate the 2D position of the part, followed by a mapping technique to transform the 2D region of interest (ROI) into the corresponding 3D point cloud region. In the point cloud processing stage, depth threshold segmentation and Euclidean clustering segmentation methods are used to separate the target part from the background and other interfering objects. The pose estimation stage uses the SAC-IA algorithm for coarse alignment, followed by an improved ICP algorithm that introduces an adaptive weighting mechanism and a global optimization strategy for fine alignment to obtain the final 6D pose of the part. The improved strategy significantly optimizes the point-pair selection and alignment process and enhances the robustness and accuracy of the algorithm. Through experimental validation on publicly available part piece datasets, the results show that the part identification and pose estimation method proposed in this study can realize fast and accurate identification and pose estimation of different shapes, non-textured, and scattered stacked parts, where the position error can reach up to 1mm and the angular error within 1°, which meets the requirements of practical applications.

Drone-based mixed reality: enhancing visualization for large-scale outdoor simulations with dynamic viewpoint adaptation using vision-based pose estimation methods

In recent years, there has been a growing interest in the integration of drones across diverse sectors, particularly within architecture, engineering, and construction (AEC). The amalgamation of drones with mixed reality (MR) stands out as a promising avenue. Proposed applications include the comparison of design and actual objects, as well as landscape simulation in urban design. A previous study successfully developed a drone-viewpoint MR system with low model dependence, leveraging general drones and methods. However, the alignment between the real and virtual worlds was contingent on predefined flight routes, limiting adaptability during MR execution. This study introduces a new model-independent drone viewpoint MR system that integrates two vision-based attitude estimation methods, enabling execution on arbitrary flight paths. Evaluation of the prototype for system latency and alignment accuracy revealed an overall latency of 3.5 s. The alignment accuracy, assessed using intersection over union, demonstrated performance equal to or surpassing the previous system. While this paper does not showcase MR content for practical use, the research lays the groundwork for advancing drone applications in the AEC field. The proposed system offers versatility for MR applications across various stages, from design to maintenance.

A Review of Human Pose Estimation Methods Based on Monocular Vision

This article is a review of the technology methods and industry applications of monocular human pose estimation, which focuses on the recent development of monocular human pose estimation. With the rapid development of computer vision and machine learning, monocular human pose estimation has become a research direction that has attracted much attention. In this article, we first introduce the relevant concepts and significance of monocular human pose estimation, explain the importance of research in this field. Then, we provide a detailed overview of the technical methods of monocular human pose estimation, including two different approaches: monocular 2D human pose estimation and monocular 3D human pose estimation. For each research method, we discuss its principles, development, and pros and cons. Next, we explore the applications of monocular human pose estimation in various fields such as human-computer interaction, autonomous driving, healthcare, and provide examples to highlight the significance of human pose estimation. Finally, we analyze the current research hotspots and challenges and also look forward to the future directions of monocular human pose estimation. This article aims to provide researchers and practitioners with a comprehensive overview and promote the application and further research of monocular human pose estimation technology.

Lightweight 2D Human Pose Estimation Based on Joint Channel Coordinate Attention Mechanism

Traditional human pose estimation methods typically rely on complex models and algorithms. Lite-HRNet can achieve an excellent performance while reducing model complexity. However, its feature extraction scale is relatively single, which can lead to lower keypoints’ localization accuracy in crowded and complex scenes. To address this issue, we propose a lightweight human pose estimation model based on a joint channel coordinate attention mechanism. This model provides a powerful information interaction channel, enabling features of different resolutions to interact more effectively. This interaction can solve the problem of human pose estimation in complex scenes and improve the robustness and accuracy of the pose estimation model. The introduction of the joint channel coordinate attention mechanism enables the model to more effectively retain key information, thereby enhancing keypoints’ localization accuracy. We also redesign the lightweight basic module using the shuffle module and the joint channel coordinate attention mechanism to replace the spatial weight calculation module in the original Lite-HRNet model. By introducing this new module, we not only improve the network calculation speed and reduce the number of parameters of the entire model, but also ensure the accuracy of the model, thereby achieving a balance between performance and efficiency. We compare this model with current mainstream methods on the COCO and MPII dataset. The experimental results show that this model can effectively reduce the number of parameters and computational complexity while ensuring high model accuracy.

Correlative Scan Matching Position Estimation Method by Fusing Visual and Radar Line Features

Millimeter-wave radar and optical cameras are one of the primary sensing combinations for autonomous platforms such as self-driving vehicles and disaster monitoring robots. The millimeter-wave radar odometry can perform self-pose estimation and environmental mapping. However, cumulative errors can arise during extended measurement periods. In particular scenes where loop closure conditions are absent and visual geometric features are discontinuous, existing loop detection methods based on back-end optimization face challenges. To address this issue, this study introduces a correlative scan matching (CSM) pose estimation method that integrates visual and radar line features (VRL-SLAM). By making use of the pose output and the occupied grid map generated by the front end of the millimeter-wave radar’s simultaneous localization and mapping (SLAM), it compensates for accumulated errors by matching discontinuous visual line features and radar line features. Firstly, a pose estimation framework that integrates visual and radar line features was proposed to reduce the accumulated errors generated by the odometer. Secondly, an adaptive Hough transform line detection method (A-Hough) based on the projection of the prior radar grid map was introduced, eliminating interference from non-matching lines, enhancing the accuracy of line feature matching, and establishing a collection of visual line features. Furthermore, a Gaussian mixture model clustering method based on radar cross-section (RCS) was proposed, reducing the impact of radar clutter points online feature matching. Lastly, actual data from two scenes were collected to compare the algorithm proposed in this study with the CSM algorithm and RI-SLAM.. The results demonstrated a reduction in long-term accumulated errors, verifying the effectiveness of the method.

Pose estimation via structure-depth information from monocular endoscopy images sequence.

Image-based endoscopy pose estimation has been shown to significantly improve the visualization and accuracy of minimally invasive surgery (MIS). This paper proposes a method for pose estimation based on structure-depth information from a monocular endoscopy image sequence. Firstly, the initial frame location is constrained using the image structure difference (ISD) network. Secondly, endoscopy image depth information is used to estimate the pose of sequence frames. Finally, adaptive boundary constraints are used to optimize continuous frame endoscopy pose estimation, resulting in more accurate intraoperative endoscopy pose estimation. Evaluations were conducted on publicly available datasets, with the pose estimation error in bronchoscopy and colonoscopy datasets reaching 1.43 mm and 3.64 mm, respectively. These results meet the real-time requirements of various scenarios, demonstrating the capability of this method to generate reliable pose estimation results for endoscopy images and its meaningful applications in clinical practice. This method enables accurate localization of endoscopy images during surgery, assisting physicians in performing safer and more effective procedures.

Deep Learning-Based 6-DoF Object Pose Estimation Considering Synthetic Dataset.

Due to the difficulty in generating a 6-Degree-of-Freedom (6-DoF) object pose estimation dataset, and the existence of domain gaps between synthetic and real data, existing pose estimation methods face challenges in improving accuracy and generalization. This paper proposes a methodology that employs higher quality datasets and deep learning-based methods to reduce the problem of domain gaps between synthetic and real data and enhance the accuracy of pose estimation. The high-quality dataset is obtained from Blenderproc and it is innovatively processed using bilateral filtering to reduce the gap. A novel attention-based mask region-based convolutional neural network (R-CNN) is proposed to reduce the computation cost and improve the model detection accuracy. Meanwhile, an improved feature pyramidal network (iFPN) is achieved by adding a layer of bottom-up paths to extract the internalization of features of the underlying layer. Consequently, a novel convolutional block attention module-convolutional denoising autoencoder (CBAM-CDAE) network is proposed by presenting channel attention and spatial attention mechanisms to improve the ability of AE to extract images' features. Finally, an accurate 6-DoF object pose is obtained through pose refinement. The proposed approach is compared to other models using the T-LESS and LineMOD datasets. Comparison results demonstrate the proposed approach outperforms the other estimation models.

DADL: Double Asymmetric Distribution Learning for head pose estimation in wisdom museum

Head pose estimation plays a pivotal role in various applications, including augmented reality and human–computer interaction within intelligent museum environments. Head pose estimation conventionally relies on hard labels. However, acquiring the “ground truth” through subjective means introduces an element of uncertainty into the labels for head pose estimation. The introduction of soft labels offers a potential remedy for this uncertainty. However, existing head pose estimation methods based on soft labels neglect the asymmetry of head pose. After careful observation, two types of asymmetry have been identified in human head pose: within angle and between angle asymmetry. Taking these two characteristics into account, we have devised a Double Asymmetric Distribution Learning (DADL) network model for the precise estimation of head pose angles. This model employs distinct soft label distribution mechanisms to capture within-angle and between-angle nuances in head pose variations. Thereby enhancing the interpretability, generalization capability, and classification accuracy of head pose estimation models. Extensive experiments were conducted on various widely recognized benchmarks, including the AFLW2000 and BIWI datasets. The results substantiate substantial advantages of our model over conventional approaches.

A Systematic Review of Recent Deep Learning Approaches for 3D Human Pose Estimation.

Three-dimensional human pose estimation has made significant advancements through the integration of deep learning techniques. This survey provides a comprehensive review of recent 3D human pose estimation methods, with a focus on monocular images, videos, and multi-view cameras. Our approach stands out through a systematic literature review methodology, ensuring an up-to-date and meticulous overview. Unlike many existing surveys that categorize approaches based on learning paradigms, our survey offers a fresh perspective, delving deeper into the subject. For image-based approaches, we not only follow existing categorizations but also introduce and compare significant 2D models. Additionally, we provide a comparative analysis of these methods, enhancing the understanding of image-based pose estimation techniques. In the realm of video-based approaches, we categorize them based on the types of models used to capture inter-frame information. Furthermore, in the context of multi-person pose estimation, our survey uniquely differentiates between approaches focusing on relative poses and those addressing absolute poses. Our survey aims to serve as a pivotal resource for researchers, highlighting state-of-the-art deep learning strategies and identifying promising directions for future exploration in 3D human pose estimation.

Yoga Pose Estimation Using Angle-Based Feature Extraction.

This research addresses the challenges of maintaining proper yoga postures, an issue that has been exacerbated by the COVID-19 pandemic and the subsequent shift to virtual platforms for yoga instruction. This research aims to develop a mechanism for detecting correct yoga poses and providing real-time feedback through the application of computer vision and machine learning (ML) techniques. This study utilized computer vision-based pose estimation methods to extract features and calculate yoga pose angles. A variety of models, including extremely randomized trees, logistic regression, random forest, gradient boosting, extreme gradient boosting, and deep neural networks, were trained and tested to classify yoga poses. Our study employed the Yoga-82 dataset, consisting of many yoga pose images downloaded from the web. The results of this study show that the extremely randomized trees model outperformed the other models, achieving the highest prediction accuracy of 91% on the test dataset and 92% in a fivefold cross-validation experiment. Other models like random forest, gradient boosting, extreme gradient boosting, and deep neural networks achieved accuracies of 90%, 89%, 90%, and 85%, respectively, while logistic regression underperformed, having the lowest accuracy. This research concludes that the extremely randomized trees model presents superior predictive power for yoga pose recognition. This suggests a valuable avenue for future exploration in this domain. Moreover, the approach has significant potential for implementation on low-powered smartphones with minimal latency, thereby enabling real-time feedback for users practicing yoga at home.

FDNet: Feature decoupling for single-stage pose estimation in complex scenes

Severe error accumulation is a key reason that makes existing multi-person pose estimation challenging in complex scenes. Single-stage pose estimation methods detect human and keypoints in parallel, avoiding serial error accumulation compared to two-stage methods. However, two distinct tasks share identical features in the existing method, which causes error accumulation. Therefore, we propose a Feature Decoupling Network (FDNet) to further extend the pipeline of single-stage methods and reconstruct task-specific features, including Feature Decoupling Module (FDM), Human Aware Loss (HAL) and Keypoint Aware Loss (KAL). FDM can adaptively perceive spatial and channel features and allocate separate feature domains for various tasks. To learn distinctive feature representation for compact human bodies, HAL and KAL measure and suppress feature similarities and distances for different human and keypoints, thus alienating feature interference. Experiments on crowded datasets show that our method is superior, outperforming the state-of-the-art method CID by 1.5%, 0.8% on OCHuman and CrowdPose.

Fiducial Objects: Custom Design and Evaluation.

Camera pose estimation is vital in fields like robotics, medical imaging, and augmented reality. Fiducial markers, specifically ArUco and Apriltag, are preferred for their efficiency. However, their accuracy and viewing angle are limited when used as single markers. Custom fiducial objects have been developed to address these limitations by attaching markers to 3D objects, enhancing visibility from multiple viewpoints and improving precision. Existing methods mainly use square markers on non-square object faces, leading to inefficient space use. This paper introduces a novel approach for creating fiducial objects with custom-shaped markers that optimize face coverage, enhancing space utilization and marker detectability at greater distances. Furthermore, we present a technique for the precise configuration estimation of these objects using multiviewpoint images. We provide the research community with our code, tutorials, and an application to facilitate the building and calibration of these objects. Our empirical analysis assesses the effectiveness of various fiducial objects for pose estimation across different conditions, such as noise levels, blur, and scale variations. The results suggest that our customized markers significantly outperform traditional square markers, marking a positive advancement in fiducial marker-based pose estimation methods.