Pose Estimation Algorithm Research Articles

LiDAR-based SLAM pose estimation via GNSS graph optimization algorithm

LiDAR simultaneous localization and mapping (SLAM) is widely used in positioning and navigation. By illuminating a series of light spots on the surface of an object, orientation and pose information is obtained. However, improving the accuracy of the pose optimization algorithm without affecting the position information is difficult. Therefore, this study combines the graph optimization algorithm and the Global Navigation Satellite System (GNSS) to optimize the coordinates of the target object in the LiDAR SLAM pose. A GNSS pose estimation algorithm is proposed to show the relationship between positioning algorithms with and without GNSS pose optimization by analyzing the deviation of the distance, level, and height laser point cloud coordinates. The results show that with GNSS pose optimization, the deviations in distance, level, and height are 99% smaller than those without GNSS pose optimization. Furthermore, we demonstrate the effectiveness of the proposed graph optimization algorithm and GNSS optimization of LiDAR SLAM. Finally, this study highlights the directions for the application of wireless communication technology in the field of LiDAR SLAM.

A Stochastic-Geometrical Framework for Object Pose Estimation Based on Mixture Models Avoiding the Correspondence Problem

Pose estimation of rigid objects is a practical challenge in optical metrology and computer vision. This paper presents a novel stochastic-geometrical modeling framework for object pose estimation based on observing multiple feature points. This framework utilizes mixture models for feature point densities in object space and for interpreting real measurements. Advantages are the avoidance to resolve individual feature correspondences and to incorporate correct stochastic dependencies in multi-view applications. First, the general modeling framework is presented, second, a general algorithm for pose estimation is derived, and third, two example models (camera and lateration setup) are presented. Numerical experiments show the effectiveness of this modeling and general algorithm by presenting four simulation scenarios for three observation systems, including the dependence on measurement resolution, object deformations and measurement noise. Probabilistic modeling utilizing mixture models shows the potential for accurate and robust pose estimations while avoiding the correspondence problem.

Design and Verification of Spacecraft Pose Estimation Algorithm using Deep Learning

This study developed a real-time spacecraft pose estimation algorithm that combined a deep learning model and the least-squares method. Pose estimation in space is crucial for automatic rendezvous docking and inter-spacecraft communication. Owing to the difficulty in training deep learning models in space, we showed that actual experimental results could be predicted through software simulations on the ground. We integrated deep learning with nonlinear least squares (NLS) to predict the pose from a single spacecraft image in real time. We constructed a virtual environment capable of mass-producing synthetic images to train a deep learning model. This study proposed a method for training a deep learning model using pure synthetic images. Further, a visual-based real-time estimation system suitable for use in a flight testbed was constructed. Consequently, it was verified that the hardware experimental results could be predicted from software simulations with the same environment and relative distance. This study showed that a deep learning model trained using only synthetic images can be sufficiently applied to real images. Thus, this study proposed a real-time pose estimation software for automatic docking and demonstrated that the method constructed with only synthetic data was applicable in space.

Basketball Fixed-point Shot Hit Prediction Based on Human Pose Estimation Algorithm

Basketball Fixed-point Shot Hit Prediction Based on Human Pose Estimation Algorithm

A Learnable Viewpoint Evolution Method for Accurate Pose Estimation of Complex Assembled Product

Balancing adaptability, reliability, and accuracy in vision technology has always been a major bottleneck limiting its application in appearance assurance for complex objects in high-end equipment production. Data-driven deep learning shows robustness to feature diversity but is limited by interpretability and accuracy. The traditional vision scheme is reliable and can achieve high accuracy, but its adaptability is insufficient. The deeper reason is the lack of appropriate architecture and integration strategies between the learning paradigm and empirical design. To this end, a learnable viewpoint evolution algorithm for high-accuracy pose estimation of complex assembled products under free view is proposed. To alleviate the balance problem of exploration and optimization in estimation, shape-constrained virtual–real matching, evolvable feasible region, and specialized population migration and reproduction strategies are designed. Furthermore, a learnable evolution control mechanism is proposed, which integrates a guided model based on experience and is cyclic-trained with automatically generated effective trajectories to improve the evolution process. Compared to the 1.69°,55.67 mm of the state-of-the-art data-driven method and the 1.28°,77.67 mm of the classic strategy combination, the pose estimation error of complex assembled product in this study is 0.23°,23.71 mm, which proves the effectiveness of the proposed method. Meanwhile, through in-depth exploration, the robustness, parameter sensitivity, and adaptability to the virtual–real appearance variations are sequentially verified.

Research on Human Posture Estimation Algorithm Based on YOLO-Pose.

In response to the numerous challenges faced by traditional human pose recognition methods in practical applications, such as dense targets, severe edge occlusion, limited application scenarios, complex backgrounds, and poor recognition accuracy when targets are occluded, this paper proposes a YOLO-Pose algorithm for human pose estimation. The specific improvements are divided into four parts. Firstly, in the Backbone section of the YOLO-Pose model, lightweight GhostNet modules are introduced to reduce the model's parameter count and computational requirements, making it suitable for deployment on unmanned aerial vehicles (UAVs). Secondly, the ACmix attention mechanism is integrated into the Neck section to improve detection speed during object judgment and localization. Furthermore, in the Head section, key points are optimized using coordinate attention mechanisms, significantly enhancing key point localization accuracy. Lastly, the paper improves the loss function and confidence function to enhance the model's robustness. Experimental results demonstrate that the improved model achieves a 95.58% improvement in mAP50 and a 69.54% improvement in mAP50-95 compared to the original model, with a reduction of 14.6 M parameters. The model achieves a detection speed of 19.9 ms per image, optimized by 30% and 39.5% compared to the original model. Comparisons with other algorithms such as Faster R-CNN, SSD, YOLOv4, and YOLOv7 demonstrate varying degrees of performance improvement.

Improved YOLOv8-Pose Algorithm for Albacore Tuna (Thunnus alalunga) Fork Length Extraction and Weight Estimation

Aiming at the problems of large statistical error and the poor real-time performance of catch weight in the ocean fishing tuna industry, an algorithm based on improved YOLOv8-Pose for albacore tuna (Thunnus alalunga) fork length extraction and weight estimation is proposed, with reference to the human body’s pose estimation algorithm. Firstly, a lightweight module constructed using a heavy parameterization technique is used to replace the backbone network, and secondly, a weighted bidirectional feature pyramid network BIFPN is utilized. Finally, the upper and lower jaw and tail feature points of the albacore tuna (Thunnus alalunga) were extracted using the key point detection algorithm, and the weight of the albacore tuna (Thunnus alalunga) was estimated based on the fitted relationship between fork length and weight. The experimental results show that the improved YOLOv8-Pose algorithm reduces the number of model parameters by 13.63% and the number of floating-point operations by 14.03% compared with the baseline model without decreasing the accuracy of the target detection and key point detection and improves the model inference speed by 374%. At the same time, it reduces the drift of the key point detection, and the error of the comparison with the actual albacore tuna (Thunnus alalunga) body weight is not more than 10%. The improved key point detection algorithm has high detection accuracy and inference speed, which provides accurate yield data for pelagic fishing and is expected to solve the existing statistical problems and improve the accuracy and real-time performance of data in the fishing industry.

Improved markerless gait kinematics measurement using a biomechanically-aware algorithm with subject-specific geometric modeling

Despite the advancements in developing markerless gait analysis systems, they still demonstrate lower accuracy compared to gold-standard systems. Hence, in this research, a novel approach is presented to improve the lower limb kinematics accuracy in markerless gait analysis. This approach refines the 3D lower-limb skeletons obtained by AI-based pose estimation algorithms in a subject-specific geometric manner, preserves skeleton links’ length, benefits from gait phases information that adds biomechanical awareness to the algorithm, and utilizes an embedded trajectory smoothing. Validation of the proposed method shows that it reduces 12.6–43.5 % of root mean square error (RMSE) and significantly improves kinematic curves’ similarity to the gold-standard ones. Results also prove the feasibility of more accurate lower limb kinematics calculation using a single (2.02–7.57° RMSE) or dual RGB-D camera (1.66–7.25° RMSE). Development of such algorithms could result in requirement of fewer cameras that deliver comparable or even superior measurement accuracy compared to multi-camera approaches.

Automatic loading method for robot guided by 3D vision for scattered and stacked shoe soles

This paper primarily introduces a method for the automated feeding of scattered stacked shoe soles using a 3D visual-guided robot. Initially, addressing issues like slow speed in pose estimation and poor robustness during the robot sorting and feeding process, we introduce an enhanced pose estimation algorithm. This algorithm combines the improved Super 4-Point Congruent Sets (Super 4PCS) with the Truncated Least Squares Semidefinite Relaxation (TEASER) algorithm, significantly boosting the speed and robustness of pose estimation during sole sorting, and achieving precise target pose estimation. Building upon this foundation, we present a sorting strategy for disordered stacked shoe soles. This strategy integrates spatial positional information of each sole, employing multi-objective decision-making and recognition algorithms to determine optimal grasping targets. Finally, an experimental platform for automated sole feeding is designed to validate the proposed method. The experimental results indicate that the pose estimation method proposed in this paper achieves an average distance error of 2.04 mm and an average angular error of 2.72°, with the robot’s average success rate in grasping reaching 97.08%. Moreover, the average processing time of the vision algorithm is only 1.34 s, demonstrating good efficiency, precision, and robustness. This method effectively meets the automated feeding needs of scattered and stacked shoe soles in the actual production processes of shoemaking enterprises.

A Parameters optimization framework for pose estimation algorithm based on point cloud

Pose estimation using point cloud data is pivotal in robotics. Despite the myriad of algorithms developed for this task, their efficiency is often contingent on optimal parameter settings. Parameters adjustment is a process that traditionally relies on extensive experience and deep understanding of the underlying algorithm. This manual, experience-driven process can hamper the adaptability and performance of pose estimation algorithms. Addressing this limitation, this paper proposes a novel framework for parameter optimization tailored to pose estimation algorithms. The framework introduces an objective function based on pose error, designed to enhance both the algorithm’s number of results and the accuracy. The methodology requires the object model and several scene point clouds with real poses of the objects.By synergistically integrating this objective function with various sampling and pruning algorithms to optimize the pose estimate algorithm, obtaining a set of superior parameters to replace default values becomes a straightforward process. Experimental assessments were conducted on the pose estimation algorithm implemented in Halcon, considering two types of objects within the Industrial 3D Object Detection Dataset (ITODD). The experimental results shown that the number and accuracy of matching results for optimized parameters are better than those proposed by algorithm developers. This not only underscores the framework’s potential as an alternative to conventional manual parameter tuning but also its utility as a foundational configuration for further refinements. Ultimately, the methodology augments the foundational efficiency and versatility of pose estimation algorithms, paving the way for more adept robotic interactions with a myriad of objects.

Integration of eye-tracking and object detection in a deep learning system for quality inspection analysis

Abstract During quality inspection in manufacturing, the gaze of a worker provides pivotal information for identifying surface defects of a product. However, it is challenging to digitize the gaze information of workers in a dynamic environment where the positions and postures of the products and workers are not fixed. A robust, deep learning-based system, ISGOD (Integrated System with worker’s Gaze and Object Detection), is proposed, which analyzes data to determine which part of the object is observed by integrating object detection and eye-tracking information in dynamic environments. The ISGOD employs a six-dimensional pose estimation algorithm for object detection, considering the location, orientation, and rotation of the object. Eye-tracking data were obtained from Tobii Glasses, which enable real-time video transmission and eye-movement tracking. A latency reduction method is proposed to overcome the time delays between object detection and eye-tracking information. Three evaluation indices, namely, gaze score, accuracy score, and concentration index are suggested for comprehensive analysis. Two experiments were conducted: a robustness test to confirm the suitability for real-time object detection and eye-tracking, and a trend test to analyze the difference in gaze movement between experts and novices. In the future, the proposed method and system can transfer the expertise of experts to enhance defect detection efficiency significantly.

Form Check: Exercise Posture Correction Application

Weightlifting is a popular sport and a hobby of millions; however, the risk of injury is high, primarily when performed with incorrect form/posture due to voluntary or involuntary reasons. It results in a significant loss of strength and hinders progress. The gradual strain on tendons and ligaments causes wear and tear. This paper aims to develop a deep learning model that detects incorrect posture and provides insight into remedial steps that need to be taken to correct the same, thereby providing increased safety and efficiency. Our Form Check model detects a user's form, vector geometry of posture is evaluated and corrected. A graphical representation of the exercise is provided as feedback to the user. Object detection, pose estimation, and action recognition algorithms infer the human skeleton and recognize the exercise. The inconsistency in the exercise performed is determined and mapped to specific imperfections in the stance of users during the exercise through Graph

3D Convolutional Neural Networks based Movement Evaluation System for Gymnasts in Computer Vision Applications

This study introduces an innovative Movement Evaluation System, which utilizes state of art 3D CNNs in grassland of computer vision. This system presents a significant advancement in analyzing and assessing complex gymnastic movements, providing a comprehensive understanding of both spatial and temporal dynamics. By incorporating pose estimation algorithms, it accurately identifies body joints and positions to extract detailed spatial features. The 3D CNN further captures the temporal evolution of these features, enabling a precise analysis of the fluidity, rhythm, and synchronization of gymnastic movements over time. The effectiveness of this system is evaluated through performance metrics such as precision, recall, and accuracy. Aimed at enhancing sports science and athlete training, this research offers coaches, judges, and gymnasts a sophisticated tool for objective and standardized movement assessments. It has the potential to streamline the evaluation process in gymnastics and provide constructive feedback efficiently. The integration of 3D CNNs in this system marks a paradigm shift in utilizing advanced computer vision techniques to improve the understanding and refinement of gymnastic performances.

Leadership dynamics in musical groups: Quantifying effects of musical structure on directionality of influence in concert performance videos.

Music ensemble performance provides an ecologically valid context for investigating leadership dynamics in small group interactions. Musical texture, specifically the relative salience of simultaneously sounding ensemble parts, is a feature that can potentially alter leadership dynamics by introducing hierarchical relationships between individual parts. The present study extended previous work on quantifying interpersonal coupling in musical ensembles by examining the relationship between musical texture and leader-follower relations, operationalised as directionality of influence between co-performers' body motion in concert video recordings. It was hypothesised that the directionality of influence, indexed by Granger Causality, would be greater for 'homophonic' textures with a clear distinction between melody and accompaniment parts than for 'polyphonic' textures with less distinction between melody and accompaniment. This hypothesis was tested by using pose estimation algorithms to track instrumentalists' body movements in a string quartet and a clarinet quintet, and then applying Granger Causality analysis to their head motion to estimate directional influence between instrumentalist pairs for sections of the pieces that varied in texture. It was found that Granger Causality values were generally higher (indicating greater directionality of influence) for homophonic than polyphonic textures. Furthermore, considering melody and accompaniment instrument roles revealed more evidence for the melody instrument influencing accompanying instruments than vice versa, plus a high degree of directionality among accompanying instruments, in homophonic textures. These observed patterns of directional information flow in co-performer body motion are consistent with changing leader-follower relations depending on hierarchical relations between ensemble parts in terms of the relative salience of melodic material in the musical texture. The finding that automatic pose estimation can detect modulations of leadership dynamics in standard video recordings under naturalistic performance conditions has implications for investigating interpersonal coordination in large-scale music video datasets representing different cultural traditions, and for exploring nonverbal communication in group activities more generally.

Hindustani raga and singer classification using 2D and 3D pose estimation from video recordings

Using pose estimation with video recordings, we apply an action recognition machine learning algorithm to demonstrate the use of the movement information to classify singers and the ragas (melodic modes) they perform. Movement information is derived from a specially recorded video dataset of solo Hindustani (North Indian) raga recordings by three professional singers each performing the same nine ragas, a smaller duo dataset (one singer with tabla accompaniment) as well as recordings of concert performances by the same singers. Data is extracted using pose estimation algorithms, both 2D (OpenPose) and 3D. A two-pathway convolutional neural network structure is proposed for skeleton action recognition to train a model to classify 12-second clips by singer and raga. The model is capable of distinguishing the three singers on the basis of movement information alone. For each singer, it is capable of distinguishing between the nine ragas with a mean accuracy of 38.2% (with the most successful model). The model trained on solo recordings also proved effective at classifying duo and concert recordings. These findings are consistent with the view that while the gesturing of Indian singers is idiosyncratic, it remains tightly linked to patterns of melodic movement: indeed we show that in some cases different ragas are distinguishable on the basis of movement information alone. A series of technical challenges are identified and addressed, with code shared alongside audiovisual data to accompany the paper.

BDR6D: Bidirectional Deep Residual Fusion Network for 6D Pose Estimation

Six-dimensional (6D) pose estimation is an important branch in the field of robotics focused on enhancing the ability of robots to manipulate and grasp objects. The latest research trend in 6D pose estimation is to directly predict the positions of two-dimensional (2D) keypoints from a single red, green, and blue (RGB) image through convolutional neural networks (CNNs) and establish a corresponding relationship with the three-dimensional (3D) keypoints of the model. Then, the perspective-n-point (PnP) algorithm is used to recover the 6D pose parameters. Currently, two challenges are encountered in pose estimation based on an RGB image. On the one hand, an RGB image lacks depth information, and it is thus difficult to directly obtain the corresponding geometric object information. On the other hand, when depth information is available, it is difficult to efficiently fuse the features of the RGB image with the features of the corresponding depth image. In this paper, we propose a bidirectional depth residual fusion network with a depth prediction (DP) network to estimate the 6D poses of objects (BDR6D). The BDR6D network predicts the depth information of objects using an RGB image, converts the depth information into point cloud information, and performs feature extraction and representation together with the RGB information during the feature extraction and representation stages. Specifically, the RGB image is fed into the BDR6D network, the DP network predicts the depth information of the objects in the image, and the depth map and RGB image are input into a point cloud network (PCN) and CNN, respectively, for feature extraction and representation. We build the bidirectional depth residual (BDR) structure so that the CNN and PCN can share information during feature extraction and representation. This approach allows the two networks to use each other’s local and global information to improve feature extraction and representation. For the keypoint selection stage, we propose an effective 2D keypoint selection method that considers the appearance and geometric information of the object of interest. We evaluate the proposed method with three benchmark datasets and compare it with other 6D pose estimation algorithms. The experimental results show that our method outperforms the state-of-the-art approach. Finally, we deploy our proposed method in conjunction with the Universal Robots 5 manipulator (UR5) robot to grasp and manipulate objects. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The purpose of this paper is to solve the problem of 6D pose estimation for robot grasping. The existing RGB image-based pose estimation approach faces two challenges. On the one hand, a single RGB image lacks depth information, so that it is difficult to directly obtain the corresponding geometric object information. On the other hand, when depth information is available, it is difficult to efficiently fuse the features of the RGB image with the features of the corresponding depth image. To solve the above problems, a novel network that can predict the depth information of objects from an RGB image and fuse the depth information with the RGB information to estimate the 6D pose of objects is proposed. Furthermore, we propose an effective 2D keypoint selection method that considers the appearance and geometric information of objects of interest. We evaluate the proposed approach based on three benchmark datasets and the UR5 robot platform and verify that our method is effective.

An efficient and accurate 2D human pose estimation method using VTTransPose network

Human pose estimation is a crucial area of study in computer vision. Transformer-based pose estimation algorithms have gained popularity for their excellent performance and relatively compact parameterization. However, these algorithms often face challenges including high computational demands and insensitivity to local details. To address these problems, the Twin attention module was introduced in TransPose to improve model efficiency and reduce resource consumption. Additionally, to address issues related to insufficient joint feature representation and poor network recognition performance, the enhanced TransPose model, named VTTransPose, replaced the basic block in the third subnet with the intra-level feature fusion module V block. The performance of the proposed VTTransPose model was validated on the public datasets COCO val2017 and COCO test-dev2017. The experimental results on COCO val2017 and COCO test-dev2017 indicate that the AP evaluation index scores of the VTTransPose network proposed are 76.5 and 73.6 respectively, marking improvements of 0.4 and 0.2 over the original TransPose network. Additionally, VTTransPose exhibited a reduction of 4.8G FLOPs, 2M parameters, and approximately 40% lower memory usage during training compared to the original TransPose model. All the experimental results demonstrate that the proposed VTTransPose is more accurate, efficient, and lightweight compared to the original TransPose model.

Clinical gait analysis using video-based pose estimation: Multiple perspectives, clinical populations, and measuring change.

Gait dysfunction is common in many clinical populations and often has a profound and deleterious impact on independence and quality of life. Gait analysis is a foundational component of rehabilitation because it is critical to identify and understand the specific deficits that should be targeted prior to the initiation of treatment. Unfortunately, current state-of-the-art approaches to gait analysis (e.g., marker-based motion capture systems, instrumented gait mats) are largely inaccessible due to prohibitive costs of time, money, and effort required to perform the assessments. Here, we demonstrate the ability to perform quantitative gait analyses in multiple clinical populations using only simple videos recorded using low-cost devices (tablets). We report four primary advances: 1) a novel, versatile workflow that leverages an open-source human pose estimation algorithm (OpenPose) to perform gait analyses using videos recorded from multiple different perspectives (e.g., frontal, sagittal), 2) validation of this workflow in three different populations of participants (adults without gait impairment, persons post-stroke, and persons with Parkinson's disease) via comparison to ground-truth three-dimensional motion capture, 3) demonstration of the ability to capture clinically relevant, condition-specific gait parameters, and 4) tracking of within-participant changes in gait, as is required to measure progress in rehabilitation and recovery. Importantly, our workflow has been made freely available and does not require prior gait analysis expertise. The ability to perform quantitative gait analyses in nearly any setting using only low-cost devices and computer vision offers significant potential for dramatic improvement in the accessibility of clinical gait analysis across different patient populations.

An overview of Graph Convolutional Networks in skeleton-based action recognition

The field of research is currently focused on human activity recognition. Hence, numerous pertinent literature reviews have expounded upon the multifaceted nature of data, the process of selecting feature vectors, and the advantages and disadvantages of classification networks. Graph Convolutional Networks (GCNs) have demonstrated significant efficacy in the domain of human action recognition. In recent years, with the rapid development of 3D skeleton data collection, a plethora of studies in action recognition based on skeleton data have emerged. Skeleton data consists of three-dimensional coordinates of multiple spatiotemporal skeletal joints, making it an effective representation of kinematics. It can be easily acquired through low-cost depth sensors and also directly extracted from two-dimensional images using video-based pose estimation algorithms, attracting widespread attention. As relational networks continue to evolve, GCNs have been applied to various fields, including human action recognition. GCNs have demonstrated significant advantages in feature extraction from skeleton data. However, using GCNs alone may have various limitations. Therefore, in recent years, many enhancement measures for GCNs have emerged. This review aims to summarize the research achievements of Graph Convolutional Network improvements in the field of human action recognition in recent years. It intends to assist future researchers in quickly organizing their research ideas, facilitating the generation of new results.

Adaptive Pose Estimation Algorithm Based on Point Pair Feature

Adaptive Pose Estimation Algorithm Based on Point Pair Feature