Precise Pose Estimation Research Articles

Pose estimation of bolster spring based on projection roundness and genetic algorithm in narrow space

Precisely identifying the pose of the bolster spring is vital for its effective disengagement. Addressing the challenge of detecting various positional arrangements of the bolster spring within the constrained space of a bogie frame, a method of estimating the pose of bolster spring based on projected roundness and genetic algorithm is proposed. This approach not only reduces the computational complexity, but also improves the precision of pose estimation. Firstly, Statistical filtering and Random Sample Consensus (RANSAC) are employed for the segmentation of the external spring point cloud and for preliminary orientation estimation. Subsequently, the convergence performance of the genetic algorithm is enhanced through optimization of the selection operator, implementation of an adaptive genetic probability adjustment strategy, and refinement of the encoding method. Finally, an evaluation function based on the projected roundness of the external spring is developed and integrated with an improved adaptive genetic algorithm (IAGA) to achieve precise pose estimation of the bolster spring. Experimental results indicate that the proposed method significantly improves pose estimation accuracy compared to traditional approaches, with an additional time cost of less than 1.5 s. The method achieves a RMSE of 1.71 mm for position and 0.84° for orientation, providing an efficient and accurate pose estimation solution for engineering applications.

High Precision Pose Estimation for Uncooperative Targets Based on Monocular Vision and 1D Laser Fusion

High Precision Pose Estimation for Uncooperative Targets Based on Monocular Vision and 1D Laser Fusion

An Object-Centric Hierarchical Pose Estimation Method Using Semantic High-Definition Maps for General Autonomous Driving.

To achieve Level 4 and above autonomous driving, a robust and stable autonomous driving system is essential to adapt to various environmental changes. This paper aims to perform vehicle pose estimation, a crucial element in forming autonomous driving systems, more universally and robustly. The prevalent method for vehicle pose estimation in autonomous driving systems relies on Real-Time Kinematic (RTK) sensor data, ensuring accurate location acquisition. However, due to the characteristics of RTK sensors, precise positioning is challenging or impossible in indoor spaces or areas with signal interference, leading to inaccurate pose estimation and hindering autonomous driving in such scenarios. This paper proposes a method to overcome these challenges by leveraging objects registered in a high-precision map. The proposed approach involves creating a semantic high-definition (HD) map with added objects, forming object-centric features, recognizing locations using these features, and accurately estimating the vehicle's pose from the recognized location. This proposed method enhances the precision of vehicle pose estimation in environments where acquiring RTK sensor data is challenging, enabling more robust and stable autonomous driving. The paper demonstrates the proposed method's effectiveness through simulation and real-world experiments, showcasing its capability for more precise pose estimation.

Multi-mode vehicle pose estimation under different GNSS conditions

The integrated navigation system combining the global navigation satellite system (GNSS) and inertial navigation system (INS) is a crucial method for pose estimation in the field of autonomous driving technologies. Nevertheless, the accuracy of pose estimation is severely compromised when GNSS signals are obstructed or disrupted. To address this issue, this study introduces a multi-mode pose estimation framework designed to ensure accurate pose estimation even under unstable GNSS conditions. By integrating vehicle kinematics model that considers steering characteristics (VKMSC) and the convolutional neural network-long short-term memory (CNN-LSTM) neural network (NN) model into various estimation modes, the framework enhances the robustness of the integrated navigation system against signal interference. The system dynamically selects the optimal estimation strategy based on the degree of GNSS signal disruption. The proposed method has been validated through real-vehicle experiments, which demonstrate its efficacy in providing precise pose estimation across a spectrum of interference scenarios. Under the multipath and non-line-of-sight (MP/NLOS) mode, compared to the integrated navigation system and the fusion of traditional vehicle kinematic models, the proposed method improved positional estimation accuracy by 61.8 % and 19.7 %, respectively. In GNSS outage mode, the proposed method increased the estimation accuracy by 36.5 % and 12.0 %, respectively, compared to the INS navigation system assisted by the VKMSC and CNN-LSTM network model. The proposed method effectively reduces pose estimation errors in the integrated navigation system during interference and suppresses data fluctuations, thereby enhancing the system's precision and robustness.

Virtual Fitness Trainer Using AI: A Research Paper

Abstract: Our research delves into the creation of a Virtual Fitness Trainer using Artificial Intelligence (AI) to offer personalized workout guidance and real-time feedback. This project harnesses advanced computer vision, machine learning, and natural language processing techniques to develop an intelligent system capable of accurately detecting human poses, tracking exercise repetitions, and providing corrective feedback. Utilizing Python's OpenCV library to capture live webcam feeds, processed by MediaPipe's BlazePose tool for precise pose estimation, our application employs a novel topology with 33 keypoints, enhancing the accuracy of body movement analysis. The frontend interface, designed with Flask, HTML, CSS, and Bootstrap, ensures an intuitive user experience, allowing selection from various exercises such as squats, curls, jumping jacks, push-ups, lateral raises, and pull-ups. Each exercise page offers detailed instructions and demonstration videos for correct technique execution. The system processes live video streams frame-by-frame, converting them into formats suitable for pose analysis and accuracy assessment. BlazePose tracks user movements in real-time, displaying a 33-keypoint exoskeleton via OpenCV, aiding in understanding body movements and tracking progress through repetition counts. The system also provides real-time feedback on exercise form, suggesting corrective actions to optimize the workout and minimize injury risks. Our Virtual Fitness Trainer addresses the limitations of existing fitness apps by enabling accurate, independent home workouts, bridging the gap left by the absence of gyms and personal trainers. It highlights AI's potential in fitness, with future advancements expected to enhance gesture recognition, adaptive workout plans, and integration with biometric data from wearables. As AI algorithms evolve, the Virtual Fitness Trainer aims to become indispensable for achieving health and wellness goals, offering a convenient, engaging, and supportive platform.

Edge Computing Framework for Enhanced Robotic Adaptivity in Line-Less Mobile Assembly Systems

In response to the shortage of skilled workers and the increasing diversity of products and variants, integrating adaptive and flexible mobile manipulators into line-less mobile assembly systems (LMAS) stands out as a promising solution. To fully benefit from these robotic systems in LMAS, offloading computationally intensive tasks from mobile manipulators to edge computing systems is essential. This offloading of computational tasks not only lightens the processing load on individual robots but also enhances AI based perception, improving the overall system capabilities in dynamic manufacturing environments. Achieving these goals requires a robotic edge computing framework that serves as the key to flexible and adaptive mobile manipulation in LMAS. To construct such an edge computing framework, a comprehensive set of building blocks is required. These building blocks include a powerful edge system with high processing power, sensor-equipped mobile manipulators with the ability to perceive the environment, and a robust edge-to-robot connectivity. A comprehensive software stack, including software modules for localization and navigation in dynamic assembly environments, AI perception for precise pose estimation of assembly components, and motion planning for interaction between manipulators and assembly components, is crucial. Furthermore, virtualization techniques, a comprehensive deployment strategy, and a detailed description of robot hardware, site, and resources are essential. The proposed edge computing framework presents a solution that addresses mobile manipulators in LMAS and paves the way towards advanced industrial automation. Through implementation in a research assembly environment, the realized proof-of-concept showcased the feasibility of the proposed edge computing framework in a real-world pick-and-place scenario, highlighting its potential to enhance adaptivity and flexibility. There is a potential to refine and expand the framework to accommodate a broader range of industrial applications and streamline diverse manufacturing processes through the integration of mobile manipulators and edge computing within industrial settings.

Robust depth-verified RGB-D visual odometry with structural regularities for indoor environments

This paper proposes an RGB-D visual odometry method that leverages point, line, plane features and Manhattan structures to achieve robust frame tracking and precise pose estimation, especially in textureless scenes. A validation method is introduced that ensures accurate frame-to-frame rotation estimation by comparing rotation angles computed from multiple Manhattan structures. Depth verification methods involving parameter fitting and outlier removal for point, line, and plane features are implemented by investigating the covariance of sensor depth measurements. We also employ local bundle adjustment in the local mapping thread to refine keyframe poses and landmarks. Comprehensive ablation studies confirm the effectiveness of our contributions. Experimental results on public datasets demonstrate that our method achieves obvious advantages in accuracy and robustness while maintaining real-time performance.

A Vision-Based Pose Estimation of a Non-Cooperative Target Based on a Self-Supervised Transformer Network

In the realm of non-cooperative space security and on-orbit service, a significant challenge is accurately determining the pose of abandoned satellites using imaging sensors. Traditional methods for estimating the position of the target encounter problems with stray light interference in space, leading to inaccurate results. Conversely, deep learning techniques require a substantial amount of training data, which is especially difficult to obtain for on-orbit satellites. To address these issues, this paper introduces an innovative binocular pose estimation model based on a Self-supervised Transformer Network (STN) to achieve precise pose estimation for targets even under poor imaging conditions. The proposed method generated simulated training samples considering various imaging conditions. Then, by combining the concepts of convolutional neural networks (CNN) and SIFT features for each sample, the proposed method minimized the disruptive effects of stray light. Furthermore, the feedforward network in the Transformer employed in the proposed method was replaced with a global average pooling layer. This integration of CNN’s bias capabilities compensates for the limitations of the Transformer in scenarios with limited data. Comparative analysis against existing pose estimation methods highlights the superior robustness of the proposed method against variations caused by noisy sample sets. The effectiveness of the algorithm is demonstrated through simulated data, enhancing the current landscape of binocular pose estimation technology for non-cooperative targets in space.

Lidar Pose Tracking of a Tumbling Spacecraft Using the Smoothed Normal Distribution Transform

Lidar sensors enable precise pose estimation of an uncooperative spacecraft in close range. In this context, the iterative closest point (ICP) is usually employed as a tracking method. However, when the size of the point clouds increases, the required computation time of the ICP can become a limiting factor. The normal distribution transform (NDT) is an alternative algorithm which can be more efficient than the ICP, but suffers from robustness issues. In addition, lidar sensors are also subject to motion blur effects when tracking a spacecraft tumbling with a high angular velocity, leading to a loss of precision in the relative pose estimation. This work introduces a smoothed formulation of the NDT to improve the algorithm’s robustness while maintaining its efficiency. Additionally, two strategies are investigated to mitigate the effects of motion blur. The first consists in un-distorting the point cloud, while the second is a continuous-time formulation of the NDT. Hardware-in-the-loop tests at the European Proximity Operations Simulator demonstrate the capability of the proposed methods to precisely track an uncooperative spacecraft under realistic conditions within tens of milliseconds, even when the spacecraft tumbles with a significant angular rate.

Effectual pre-processing with quantization error elimination in pose detector with the aid of image-guided progressive graph convolution network (IGP-GCN) for multi-person pose estimation

Multi-person pose estimation (MPE) remains a significant and intricate issue in computer vision. This is considered the human skeleton joint identification issue and resolved by the joint heat map regression network lately. Learning robust and discriminative feature maps is essential for attaining precise pose estimation. Even though the present methodologies established vital progression via feature map’s interlayer fusion and intralevel fusion, some studies show consideration for the combination of these two methodologies. This study focuses upon three phases of pre-processing stages like occlusion elimination, suppression strategy, and heat map methodology to lessen noise within the database. Subsequent to pre-processing errors will be eliminated by employing the quantization phase by embracing the pose detector. Lastly, Image-Guided Progressive Graph Convolution Network (IGP-GCN) has been built for MPE. This IGP-GCN consistently learns rich fundamental spatial information by merging features inside the layers. In order to enhance high-level semantic information and reuse low-level spatial information for correct keypoint representation, this also provides hierarchical connections across feature maps of the same resolution for interlayer fusion. Furthermore, a missing connection between the output high level information and low-level information was noticed. For resolving the issue, the effectual shuffled attention mechanism has been proffered. This shuffle intends to support the cross-channel data interchange between pyramid feature maps, whereas attention creates a trade-off between the high level and low-level representations of output features. This proffered methodology can be called Occlusion Removed_Image Guided Progressive Graph Convolution Network (OccRem_IGP-GCN), and, thus, this can be correlated with the other advanced methodologies. The experimental outcomes exhibit that the OccRem_IGP-GCN methodology attains 98% of accuracy, 93% of sensitivity, 92% of specificity, 88% of f1-score, 42% of relative absolute error, and 30% of mean absolute error.

Transformer-Based Self-Supervised Monocular Depth and Visual Odometry

Self-supervised monocular depth and visual odometry (VO) are often cast as coupled tasks. Accurate depth contributes to precise pose estimation and vice versa. Existing architectures typically exploit stacking convolution layers and long short-term memory (LSTM) units to capture long-range dependencies. However, their intrinsic locality hinders the model from getting the expected performance gain. In this article, we propose a Transformer-based architecture, named Transformer-based self-supervised monocular depth and VO (TSSM-VO), to tackle these problems. It comprises two main components: 1) a depth generator that leverages the powerful capability of multihead self-attention (MHSA) on modeling long-range spatial dependencies and 2) a pose estimator built upon a Transformer to learn long-range temporal correlations of image sequences. Moreover, a new data augmentation loss based on structural similarity (SSIM) is introduced to constrain further the structural similarity between the augmented depth and the augmented predicted depth. Rigorous ablation studies and exhaustive performance comparison on the KITTI and Make3D datasets demonstrate the superiority of TSSM-VO over other self-supervised methods. We expect that TSSM-VO would enhance the ability of intelligent agents to understand the surrounding environments.

Aasna: Kinematic Yoga Posture Detection And Correction System Using CNN

Yoga is a very popular form of exercise that originated in India that has numerous benefits for the mind and body. According to recent statistics, there are over 300 million yoga practitioners worldwide, with the number of yoga instructors increasing annually. However, incorrect yoga postures can lead to injuries and health complications. This highlights the importance of correct yoga posture and the need for a system that can detect and correct improper poses. This abstract presents a yoga posture detection and correction system designed using OpenCV for computer vision, and kinematic representation of the human body considering 17 points mapped on the human body, utilizing the tf-pose estimation algorithm for precise pose estimation. The system also includes a convolutional neural network (CNN) model developed using the Keras API and trained on the TensorFlow platform's MoveNet architecture for handling training of the model. The MoveNet pose estimation module has been used to detect keypoints of the human body which achieved an accuracy of 99.88%. The system works by live capturing of the yoga practitioner using a camera, extracting the key features of the pose, and comparing them with a trained data model of known yoga poses. If the pose is incorrect, the system provides real-time feedback to correct the pose.

Precise pose estimation of the NASA Mars 2020 Perseverance rover through a stereo‐vision‐based approach

AbstractVisual Odometry (VO) is a fundamental technique to enhance the navigation capabilities of planetary exploration rovers. By processing the images acquired during the motion, VO methods provide estimates of the relative position and attitude between navigation steps with the detection and tracking of two‐dimensional (2D) image keypoints. This method allows one to mitigate trajectory inconsistencies associated with slippage conditions resulting from dead‐reckoning techniques. We present here an independent analysis of the high‐resolution stereo images of the NASA Mars 2020 Perseverance rover to retrieve its accurate localization on sols 65, 66, 72, and 120. The stereo pairs are processed by using a 3D‐to‐3D stereo‐VO approach that is based on consolidated techniques and accounts for the main nonlinear optical effects characterizing real cameras. The algorithm is first validated through the analysis of rectified stereo images acquired by the NASA Mars Exploration Rover Opportunity, and then applied to the determination of Perseverance's path. The results suggest that our reconstructed path is consistent with the telemetered trajectory, which was directly retrieved onboard the rover's system. The estimated pose is in full agreement with the archived rover's position and attitude after short navigation steps. Significant differences (~10–30 cm) between our reconstructed and telemetered trajectories are observed when Perseverance traveled distances larger than 1 m between the acquisition of stereo pairs.

PerUnet: Deep Signal Channel Attention in Unet for WiFi-Based Human Pose Estimation

Traditional image-based pose estimation methods tend to perform poorly in occlusion and darkness. Thus, researchers established sensor-based pose estimation methods such as radio frequency and infrared to resolve the challenges. Nevertheless, traditional sensors remain the weakness of high cost and low flexibility. In this article, we propose the multimodal network PerUnet to establish a WiFi-based human pose estimation network. Benefitting from the powerful multihead attention mechanism and Unet-like architecture, PerUnet fuses fine-grained pose features and contextual information of WiFi channel state information (CSI) to achieve precise pose estimation. Moreover, we propose the attention-based denoising (ABD) method to overcome traditional filters’ disadvantages and help PerUnet extract pose features from CSI. To evaluate the performance of PerUnet, we establish the novel multimodal dataset Wi-Pose composed of images, CSI, and pose annotations. The experimental results demonstrate that PerUnet achieves a competitive performance of WiFi-based pose estimation on Wi-Pose. To further promote the research on WiFi-based human pose estimation, Wi-Pose has been publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NjtechCVLab/Wi-PoseDataset</uri> .

A novel encoding element for robust pose estimation using planar fiducials.

Pose estimation in robotics is often achieved using images from known and purposefully applied markers or fiducials taken by a monocular camera. This low-cost system architecture can provide accurate and precise pose estimation measurements. However, to prevent the restriction of robotic movement and occlusions of features, the fiducial markers are often planar. While numerous planar fiducials exist, the performance of these markers suffers from pose ambiguities and loss of precision under frontal observations. These issues are most prevalent in systems with less-than-ideal specifications such as low-resolution detectors, low field of view optics, far-range measurements etc. To mitigate these issues, encoding markers have been proposed in literature. These markers encode an extra dimension of information in the signal between marker and sensor, thus increasing the robustness of the pose solution. In this work, we provide a survey of these encoding markers and show that existing solutions are complex, require optical elements and are not scalable. Therefore, we present a novel encoding element based on the compound eye of insects such as the Mantis. The encoding element encodes a virtual point in space in its signal without the use of optical elements. The features provided by the encoding element are mathematically equivalent to those of a protrusion. Where existing encoding fiducials require custom software, the projected virtual point can be used with standard pose solving algorithms. The encoding element is simple, can be produced using a consumer 3D printer and is fully scalable. The end-to-end implementation of the encoding element proposed in this work significantly increases the pose estimation performance of existing planar fiducials, enabling robust pose estimation for robotic systems.

Real-Time Localization and Mapping Utilizing Multi-Sensor Fusion and Visual–IMU–Wheel Odometry for Agricultural Robots in Unstructured, Dynamic and GPS-Denied Greenhouse Environments

Autonomous navigation in greenhouses requires agricultural robots to localize and generate a globally consistent map of surroundings in real-time. However, accurate and robust localization and mapping are still challenging for agricultural robots due to the unstructured, dynamic and GPS-denied environmental conditions. In this study, a state-of-the-art real-time localization and mapping system was presented to achieve precise pose estimation and dense three-dimensional (3D) point cloud mapping in complex greenhouses by utilizing multi-sensor fusion and Visual–IMU–Wheel odometry. In this method, measurements from wheel odometry, an inertial measurement unit (IMU) and a tightly coupled visual–inertial odometry (VIO) are integrated into a loosely coupled framework based on the Extended Kalman Filter (EKF) to obtain a more accurate state estimation of the robot. In the multi-sensor fusion algorithm, the pose estimations from the wheel odometry and IMU are treated as predictions and the localization results from VIO are used as observations to update the state vector. Simultaneously, the dense 3D map of the greenhouse is reconstructed in real-time by employing the modified ORB-SLAM2. The performance of the proposed system was evaluated in modern standard solar greenhouses with harsh environmental conditions. Taking advantage of measurements from individual sensors, our method is robust enough to cope with various challenges, as shown by extensive experiments conducted in the greenhouses and outdoor campus environment. Additionally, the results show that our proposed framework can improve the localization accuracy of the visual–inertial odometry, demonstrating the satisfactory capability of the proposed approach and highlighting its promising applications in autonomous navigation of agricultural robots.

Intelligent vehicle visual pose estimation algorithm based on deep learning and parallel computing for dynamic scenes

In view of the disadvantages of the existing pose estimation algorithm, which has low real-time performance and the positioning accuracy will be greatly reduced in dynamic scene, a compound deep learning and parallel computing algorithm (DP-PE) is proposed. The detection algorithm based on deep learning is used to detect dynamic objects in the environment, and the dynamic feature points are removed before the matching of feature points to reduce the impact of dynamic objects on the positioning accuracy; A method for distinguishing “pseudo-dynamic objects” is proposed to solve the problem that the stationary vehicles and pedestrians in the environment are regarded as dynamic objects. The parallel computing framework for feature point extraction and matching is established on CPU-GPU heterogeneous platform to speed up DP-PE; In the localization part of DP-PE, we propose a 3D interior point detection strategy to achieve parallel search of map points, and the saturated linear kernel function is used to act on reprojection error to realize the parallelization of pose optimization. We verify the algorithm on KITTI dataset, the experimental results show that average speedup ratio of feature point extraction and matching is 6.5 times, and the overall computational efficiency of DP-PE is about 7 times higher than that before acceleration, which can realize high precision and efficient pose estimation in dynamic scene.

MISD-SLAM: Multimodal Semantic SLAM for Dynamic Environments

Simultaneous localization and mapping (SLAM) is one of the most essential technologies for mobile robots. Although great progress has been made in the field of SLAM in recent years, there are a number of challenges for SLAM in dynamic environments and high-level semantic scenes. In this paper, we propose a novel multimodal semantic SLAM system (MISD-SLAM), which removes the dynamic objects in the environments and reconstructs the static background with semantic information. MISD-SLAM builds three main processes: instance segmentation, dynamic pixels removal, and semantic 3D map construction. An instance segmentation network is used to provide semantic knowledge of surrounding environments in instance level. The ORB features located on the predefined dynamic objects are removed directly. In this way, MISD-SLAM effectively reduces the impact of dynamic objects to provide precise pose estimation. Then, combining multiview geometry constraint with K -means clustering algorithm, our system removes the undefined but moving pixels. Meanwhile, a 3D dense point cloud map with semantic information is reconstructed, which recovers the static background without the corruptions of dynamic objects. Finally, we evaluate MISD-SLAM by comparing to ORB-SLAM3 and the state-of-the-art dynamic SLAM systems in TUM RGB-D datasets and real-world dynamic indoor environments. The results indicate that our method significantly improves the localization accuracy and system robustness, especially in high-dynamic environments.

Cross-Domain Self-Supervised Complete Geometric Representation Learning for Real-Scanned Point Cloud Based Pathological Gait Analysis.

Accurate lower-limb pose estimation is aprerequisite of skeleton based pathological gait analysis. To achieve this goal in free-living environments for long-term monitoring, single depth sensor has been proposed in research. However, the depth map acquired from a single viewpoint encodes only partial geometric information of the lower limbs and exhibits large variations across different viewpoints. Existing off-the-shelf 3D pose tracking algorithms and public datasets for depth based human pose estimation are mainly targeted at activity recognition applications. They are relatively insensitive to skeleton estimation accuracy, especially at the foot segments. Furthermore, acquiring ground truth skeleton data for detailed biomechanics analysis also requires considerable efforts. To address these issues, we propose a novel cross-domain self-supervised complete geometric representation learning framework, with knowledge transfer from the unlabelled synthetic point clouds of full lower-limb surfaces. The proposed method can significantly reduce the number of ground truth skeletons (with only 1%) in the training phase, meanwhile ensuring accurate and precise pose estimation and capturing discriminative features across different pathological gait patterns compared to other methods.

Toward Coordination Control of Multiple Fish-Like Robots: Real-Time Vision-Based Pose Estimation and Tracking via Deep Neural Networks

Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists, for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time. This requires detecting multiple robots, estimating multi-joint postures, and tracking identities, as well as processing fast in real time. To the best of our knowledge, this challenge has not been tackled in the previous studies. In this paper, to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time, we propose a novel deep neural network-based method, named TAB-IOL. Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation, while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking. The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy, speed, and robustness with most state-of-the-art algorithms. Further, based on the precise pose estimation and tracking realized by our TAB-IOL, several formation control experiments are conducted for the group of fish-like robots. The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment. We believe our proposed method will facilitate the growth and development of related fields.