Pose Estimation System Research Articles

Autonomous Positioning for Mobile Vehicles Based on Visual‐Inertial Fusion in Challenging Dark Roadway Scenes

ABSTRACTAccurate positioning for autonomous driven underground mining vehicles (UMVs) and coal mine robots (CMRs) is indeed one of the cores in the intelligentization of coal mining. Completely different from positioning on the ground and in parking scenes, there will be great difficulties in realizing the accurate active positioning for UMVs shuttled in dark and narrow roadways. We propose an effective visual and inertial fusion positioning method for autonomous CMRs and roadheaders in challenging roadway scenarios based on the odometer‐aided inertial navigation system and visual pose estimation system. Velocity information of the odometer is adapted to restrain the error accumulation of inertial positioning based on a Kalman filter. The hybrid visual feature detection algorithm is put forward to improve the accuracy and robustness of visual observation information in a dark environment. Autonomous experiments for CMRs and roadheaders are separately performed in the narrow roadway and dark passageway to demonstrate the applicability of our localization method. The proposed approach outperforms the subsystems and existing methods in accuracy and has outstanding stability.

Discrimination between the facial gestures of vocalising and non-vocalising lemurs and small apes using deep learning

Facial expression studies in animal communication are essential. However, manual inspection methods are only practical for small datasets. Deep learning techniques can help discriminate facial configurations associated with vocalisations over large datasets. We extracted and labelled frames of different primate species, trained deep-learning models to identify key points on their faces, and computed distances between them to identify facial gestures. We used machine learning algorithms to classify vocalised and non-vocalised gestures across different species. The algorithms showed higher-than-chance correct classification rates, with some exceeding 90 %. Our work employs deep learning to map primate facial gestures and offers an innovative application of pose estimation systems. Our approach facilitates the investigation of facial repertoire across primate species and behavioural contexts, enabling comparative research in primate communication.

MCP: Multi-Chicken Pose Estimation Based on Transfer Learning.

Poultry managers can better understand the state of poultry through poultry behavior analysis. As one of the key steps in behavior analysis, the accurate estimation of poultry posture is the focus of this research. This study mainly analyzes a top-down pose estimation method of multiple chickens. Therefore, we propose the "multi-chicken pose" (MCP), a pose estimation system for multiple chickens through deep learning. Firstly, we find the position of each chicken from the image via the chicken detector; then, an estimate of the pose of each chicken is made using a pose estimation network, which is based on transfer learning. On this basis, the pixel error (PE), root mean square error (RMSE), and image quantity distribution of key points are analyzed according to the improved chicken keypoint similarity (CKS). The experimental results show that the algorithm scores in different evaluation metrics are a mean average precision (mAP) of 0.652, a mean average recall (mAR) of 0.742, a percentage of correct keypoints (PCKs) of 0.789, and an RMSE of 17.30 pixels. To the best of our knowledge, this is the first time that transfer learning has been used for the pose estimation of multiple chickens as objects. The method can provide a new path for future poultry behavior analysis.

An end-to-end learning framework for visual camera relocalization using RGB and RGB-D images

Camera relocalization plays a vital role in the realms of machine perception, robotics, and augmented reality. Direct learning methods based on structures can have a learning-based approach that can learn scene coordinates and use them for camera position estimation. However, the two-stage learning of scene coordinate regression and camera position estimation can result in some of the scene coordinate regression knowledge being lost throughout the learning process of the final pose estimation system, thereby reducing the accuracy of the pose estimation. This paper introduces an innovative end-to-end learning framework tailored for visual camera relocalization by employing both RGB and RGB-D images. Distinguished by its integration of scene coordinate regression with pose estimation into a concurrent inner and outer loop during a singular training phase, this framework notably enhances pose estimation accuracy. Engineered for flexibility, it accommodates training with or without depth cues and necessitates merely a single RGB image during testing. Empirical evaluation substantiates the proposed method’s state-of-the-art precision, attaining an average pose accuracy of 0.019 m and 0.74° on the indoor 7Scenes dataset, together with 0.162 m and 0.30° on the outdoor Cambridge Landmarks dataset.

Comparative Analysis of Human Pose Estimation Methods with Fall Detection and Smart Alarming System

The population of old age peoples are increasing day by day and it will grow up by 31% of today’s population till 2031. In many cases the old age people are prone to fall. So, old age peoples can help in various aspect by using advanced technology. We already know that fall discovery is very much critical part of senior care as fall pose a threat to their health. To this challenge, this discussion and study proposes a fall detection model comprising three crucial phases: Pose Estimation, Fall Detection, and Smart Alerting System. Using computer vision model similar as OpenPose, PoseNet, MoveNet and YOLOv7. By the help of this models, we directly track the act of senior individualities in real time and identifies fall incidents grounded by abnormal changes in posture. When fall detect, the WhatsApp Alert message with fall detected frame and phone call will be received by Caregiver or their family members to take urgent actions. In case of fall, one can prevent from severe injuries which could be caused due to delay in mediation. These proposed models will enhance the safety of old age population.

Enhancement and Evaluation of Workout Postures through AI-Powered Pose Estimation

Strength training and yoga carry injury risks for beginners, demanding tailored feedback. Advances in deep learning present real-world applications, yet its adoption in fitness remains limited. This study explores human pose estimation for workout feedback, emphasizing technique-related issues. An analytical approach involves developing a system for exercise and filming detection and evaluating technique aspects.[3] The research aims to establish a foundation for an exercise feedback application through a comparative study of two deep learning models. Testing focuses on ten exercise categories. A comprehensive review of pose estimation systems evaluates their effectiveness, demonstrating promise for workout analysis. Despite limitations, further exploration is warranted for this promising application. Keywords- Visual Perception of Computers, Advanced Convolutional Neural Networks, Body Position Assessment, Exercise Identification.

Invariant Kalman Filter Design for Securing Robust Performance of Magnetic–Inertial Integrated Navigation System under Measurement Uncertainty

This study proposes an enhanced integration algorithm that combines the magnetic field-based positioning system (MPS—Magnetic Pose Estimation System) with an inertial system with the advantage of an invariant filter structure. Specifically, to mitigate the nonlinearity of the propagation model and perturbing effect from the estimated uncertainty, the formulation of the invariant Kalman filter was derived in detail. Then, experiments were conducted to validate the algorithm with an unmanned vehicle equipped with an IMU and MPS receiver. As a result, the navigation performance of the IEKF-based inertial and magnetic field integration system was presented and compared with the conventional Kalman filter results. Furthermore, the convergence and navigation performance were evaluated in the presence of state variable initialization errors. The findings indicate that the inertial and magnetic field coupled with the IEKF outperforms the typical KF approach, particularly when dealing with initial estimate uncertainties.

Trading-Off Safety with Agility Using Deep Pose Error Estimation and Reinforcement Learning for Perception-Driven UAV Motion Planning

Navigation and planning for unmanned aerial vehicles (UAVs) based on visual-inertial sensors has been a popular research area in recent years. However, most visual sensors are prone to high error rates when exposed to disturbances such as excessive brightness and blur, which can lead to catastrophic performance drops in perception and motion planning systems. This study proposes a novel framework to address the coupled perception-planning problem in high-risk environments. This achieved by developing algorithms that can automatically adjust the agility of the UAV maneuvers based on the predicted error rate of the pose estimation system. The fundamental idea behind our work is to demonstrate that highly agile maneuvers become infeasible to execute when visual measurements are noisy. Thus, agility should be traded-off with safety to enable efficient risk management. Our study focuses on navigating a quadcopter through a sequence of gates on an unknown map, and we rely on existing deep learning methods for visual gate-pose estimation. In addition, we develop an architecture for estimating the pose error under high disturbance visual inputs. We use the estimated pose errors to train a reinforcement learning agent to tune the parameters of the motion planning algorithm to safely navigate the environment while minimizing the track completion time. Simulation results demonstrate that our proposed approach yields significantly fewer crashes and higher track completion rates compared to approaches that do not utilize reinforcement learning.

IoT-enabled Biometric Security: Enhancing Smart Car Safety with Depth-based Head Pose Estimation

Advanced Driver Assistance Systems (ADAS) are experiencing higher levels of automation, facilitated by the synergy among various sensors integrated within vehicles, thereby forming an Internet of Things (IoT) framework. Among these sensors, cameras have emerged as valuable tools for detecting driver fatigue and distraction. This study introduces HYDE-F, a Head Pose Estimation (HPE) system exclusively utilizing depth cameras. HYDE-F adeptly identifies critical driver head poses associated with risky conditions, thus enhancing the safety of IoT-enabled ADAS. The core of HYDE-F’s innovation lies in its dual-process approach: it employs a fractal encoding technique and keypoint intensity analysis in parallel. These two processes are then fused using an optimization algorithm, enabling HYDE-F to blend the strengths of both methods for enhanced accuracy. Evaluations conducted on a specialized driving dataset, Pandora, demonstrate HYDE-F’s competitive performance compared to existing methods, surpassing current techniques in terms of average Mean Absolute Error (MAE) by nearly 1 ∘ . Moreover, case studies highlight the successful integration of HYDE-F with vehicle sensors. Additionally, HYDE-F exhibits robust generalization capabilities, as evidenced by experiments conducted on standard laboratory-based HPE datasets, i.e., Biwi and ICT-3DHP databases, achieving an average MAE of 4.9 ∘ and 5 ∘ , respectively.

Hip contact forces can be predicted with a neural network using only synthesised key points and electromyography in people with hip osteoarthritis

ObjectiveTo develop and validate a neural network to estimate hip contact forces (HCF), and lower body kinematics and kinetics during walking in individuals with hip osteoarthritis (OA) using synthesised anatomical key points and electromyography. To assess the capability of the neural network to detect directional changes in HCF resulting from prescribed gait modifications. DesignA calibrated electromyography-informed neuromusculoskeletal model was used to compute lower body joint angles, moments, and HCF for 17 participants with mild-to-moderate hip OA. Anatomical key points (e.g., joint centres) were synthesised from marker trajectories and augmented with bias and noise expected from computer vision-based pose estimation systems. Temporal convolutional and long short-term memory neural networks were trained using leave-one-subject-out validation to predict neuromusculoskeletal modelling outputs from the synthesised key points and measured electromyography data from 5 hip-spanning muscles. ResultsHCF was predicted with an average error of 13.4±7.1% of peak force. Joint angles and moments were predicted with an average root-mean-square-error of 5.3 degrees and 0.10 Nm/kg, respectively. The NN could detect changes in peak HCF that occur due to gait modifications with good agreement with neuromusculoskeletal modelling (r2 = 0.72) and a minimum detectable change of 9.5%. ConclusionThe developed neural network predicted HCF and lower body joint angles and moments in individuals with hip OA using noisy synthesised key point locations with acceptable errors. Changes in HCF magnitude due to gait modifications were predicted with high accuracy. These findings have important implications for implementation of load-modification based gait retraining interventions for people with hip OA in a natural environment (i.e., home, clinic).

Hardware/Software Co-Design of a Feature-Based Satellite Pose Estimation System

Hardware/Software Co-Design of a Feature-Based Satellite Pose Estimation System

MDPose: Human Skeletal Motion Reconstruction Using WiFi Micro-Doppler Signatures

Motion tracking systems based on optical sensors typically suffer from poor lighting conditions, occlusion, limited coverage, and may raise privacy concerns. More recently, radiofrequency (RF) based approaches using commercial WiFi devices have emerged which offer low-cost ubiquitous sensing whilst preservin privacy. However, RF sensing systems typically output range-Doppler maps, time-frequency spectrograms, cross-range plots etc which cannot represent human motion intuitively and usually requires further processing. In this study, we propose MDPose, a novel framework for human skeletal motion reconstruction base on WiFi micro-Doppler signatures. MDPose provides an effective solution to represent human activity by reconstructing a skeleton model with 17 key points, which can assist with the interpretation of conventional RF sensing outputs in a more understandable way. Specifically, MDPose is implemented over three sequential stage to address a series of challenges: First, a denoising algorithm is employed to remove any unwanted noise that may affect feature extraction and enhance weak Doppler measurements. Secondly, a convolutional neural network (CNN)-recurrent neural network (RNN) architecture is applied to learn temporal spatial dependenc from clean micro-Doppler signatures and restore velocity information to key points under the supervision of the motion capture (Mocap) system. Finally, a pose optimisation mechanism based on learning optimisation vectors is employed to estimate the initial state of the skeleton and to limit additional errors. We hav conducted a comprehensive set of tests in a variety of environments using numerous subjects with a single receiver radar system to demonstrate the performance of MDPose, and report 29.4mm mean absolute error over all key points positions on several common daily activities, which has performance comparable to that of state-ofthe- art RF-based pose estimation systems.

Improving Gait Analysis Techniques with Markerless Pose Estimation Based on Smartphone Location.

Marker-based 3D motion capture systems, widely used for gait analysis, are accurate but have disadvantages such as cost and accessibility. Whereas markerless pose estimation has emerged as a convenient and cost-effective alternative for gait analysis, challenges remain in achieving optimal accuracy. Given the limited research on the effects of camera location and orientation on data collection accuracy, this study investigates how camera placement affects gait assessment accuracy utilizing five smartphones. This study aimed to explore the differences in data collection accuracy between marker-based systems and pose estimation, as well as to assess the impact of camera location and orientation on accuracy in pose estimation. The results showed that the differences in joint angles between pose estimation and marker-based systems are below 5°, an acceptable level for gait analysis, with a strong correlation between the two datasets supporting the effectiveness of pose estimation in gait analysis. In addition, hip and knee angles were accurately measured at the front diagonal of the subject and ankle angle at the lateral side. This research highlights the significance of careful camera placement for reliable gait analysis using pose estimation, serving as a concise reference to guide future efforts in enhancing the quantitative accuracy of gait analysis.

Rein-SLAM: Narrow the Gaps Between the Matching Task and SLAM System

Image matching task and feature-based SLAM (Simultaneous Localization And Mapping) systems both rely on feature points to complete pixel association, which provide an important support and foundation for the perception and integration of unmanned vehicles in the industrial park environments. In this article, we point out there exist associations and gaps between the matching task and SLAM systems. Based on the feature extraction network for matching task, we propose an effective training strategy in reference to the operating mode of SLAM systems, so that the feature extraction network can better adapt to the SLAM system for pose estimation and mapping. The proposed method only needs one sequence in the same scene for training, and the whole training process can be completed in only one epoch which takes no more than 20 minutes on a RTX 3080 GPU. We conduct experiments on HPatches dataset (matching task dataset), two public SLAM datasets: KITTI dataset for outdoor scenes and Euroc dataset for indoor scenes, and our collected dataset which contains several challenging scenes. Our experiments show that our method can not only effectively improve the performance of the extraction network on the original matching task, but also greatly improve the performance of its integrated SLAM system. Furthermore, our method can be easily transplanted to other image matching networks, so that they can be better used in SLAM systems and narrow the gaps between the matching tasks and SLAM systems.

DIG-SLAM: an accurate RGB-D SLAM based on instance segmentation and geometric clustering for dynamic indoor scenes

Simultaneous localization and mapping (SLAM) has emerged as a critical technology enabling robots to navigate in unknown environments, drawing extensive attention within the robotics research community. However, traditional visual SLAM ignores the presence of dynamic objects in indoor scenes, and dynamic point features of dynamic objects can lead to incorrect data correlation, making the traditional visual SLAM is difficult to accurately estimate the camera’s pose when the objects in the scenes are moving. Using only point features cannot fully extract geometric information in dynamic indoor scenes, reducing the system’s robustness. To solve this problem, we develop a RGB-D SLAM system called DIG-SLAM. Firstly, the objects’ contour regions are extracted using the YOLOv7 instance segmentation method, serving as a prerequisite for determining dynamic objects and constructing a semantic information map. Meanwhile, the line features are extracted using the line segment detector algorithm, and the redundant line features are optimized via K-means clustering. Secondly, moving consistency checks combined with instance partitioning determine dynamic regions, and the point and line features of the dynamic regions are removed. Finally, the combination of static line features and point features optimizes the camera pose. Meanwhile, a static semantic octree map is created to provide richer and higher-level scene understanding and perception capabilities for robots or autonomous systems. The experimental results on the Technische Universität München dataset show that the average absolute trajectory error of the developed DIG-SLAM is reduced by 28.68% compared with the dynamic semantic SLAM. Compared with other dynamic SLAM methods, the proposed system shows better camera pose estimation accuracy and system’s robustness in dynamic indoor environments and better map building in real indoor scenes.

Two-Phase Positioning System Based on the Fusion of Wi-Fi Signal Strength and Pose Estimation

Due to the growing importance of positioning in recent years, global positioning system has become widely used. However, indoor positioning presents significant challenges due to multipath interference, which can cause large signal fluctuations and high positioning errors for signal-based approaches. Recent advances in deep learning have enabled researchers to estimate people's positions and locate them indoors using image recognition technology, but identification of individuals remains challenging. Therefore, we propose a highly accurate indoor positioning approach using Wi-Fi signals and images. This research employs smartphones to measure the signal strength of three Wi-Fi base stations' 2.4 and 5 GHz frequency bands. We then use machine learning techniques to estimate individuals' positions with coarse accuracy. Subsequently, we analyze the positions of individuals in surveillance camera images. We use OpenPose, an open-source multiperson pose estimation system, and propose a foot position extraction algorithm to extract the foot positions of individuals in the image. We obtain the coordinates of the people in the image through direct linear transformation and machine learning. Finally, we match the coordinates obtained from the two techniques to identify the user's device. Experimental results show that the proposed method achieves higher accuracy, with an average error of 0.43 meters.

OTOT: An online training and offline testing system for 6D Object Pose Estimation

OTOT: An online training and offline testing system for 6D Object Pose Estimation

Comparison of a single-view image-based system to a multi-camera marker-based system for human static pose estimation

The purpose of this study was to compare human static pose estimation data measured with a single-view image-based system and a multi-camera marker-based system. Thirty participants (20 male/10 female, mean ± standard deviation 29.1 ± 10.0 years old, 1.75 ± 0.10 m tall, 79.1 ± 18.0 kg) performed six repetitions each of static holds of arm-raises and squats, in a different orientation for each repetition. These trials were captured simultaneously with a 120-Hz 12-camera marker-based system and a variable-frequency single-view image-based system. Data for each trial were time-synchronized between the two systems using a near-infrared LED-light that was visible to both systems. Discrete measurements of bilateral shoulder angles during arm-raises and bilateral knee angles during squats were compared between the systems using Bland-Altman plots and descriptive statistics. Pearson correlation coefficients were calculated, comparing the participant trial mean values across systems. Finally, a two-way ANOVA was used to examine whether participant orientation in the capture volume significantly affected either system. Biases for discrete measurements ranged in magnitude from 1.3 to 1.9°, and standard deviations of the differences between systems ranged from 2.4 to 4.7°. Pearson correlation coefficients were all above 0.97, and the ANOVA was unable to find a statistically significant orientation effect for either system. Thus, the marker-based and image-based systems produced similar measurements of static shoulder and knee angles. Future work should examine more complex measurements using volumetric scan-based models and also investigate the ability of single-view image-based systems to measure dynamic movements.

High Speed and Accuracy of Animation 3D Pose Recognition Based on an Improved Deep Convolution Neural Network

Pose recognition in character animations is an important avenue of research in computer graphics. However, the current use of traditional artificial intelligence algorithms to recognize animation gestures faces hurdles such as low accuracy and speed. Therefore, to overcome the above problems, this paper proposes a real-time 3D pose recognition system, which includes both facial and body poses, based on deep convolutional neural networks and further designs a single-purpose 3D pose estimation system. First, we transformed the human pose extracted from the input image to an abstract pose data structure. Subsequently, we generated the required character animation at runtime based on the transformed dataset. This challenges the conventional concept of monocular 3D pose estimation, which is extremely difficult to achieve. It can also achieve real-time running speed at a resolution of 384 fps. The proposed method was used to identify multiple-character animation using multiple datasets (Microsoft COCO 2014, CMU Panoptic, Human3.6M, and JTA). The results indicated that the improved algorithm improved the recognition accuracy and performance by approximately 3.5% and 8–10 times, respectively, which is significantly superior to other classic algorithms. Furthermore, we tested the proposed system on multiple pose-recognition datasets. The 3D attitude estimation system speed can reach 24 fps with an error of 100 mm, which is considerably less than that of the 2D attitude estimation system with a speed of 60 fps. The pose recognition based on deep learning proposed in this study yielded surprisingly superior performance, proving that the use of deep-learning technology for image recognition has great potential.

Automatic feature selection for performing Unit 2 of vault in wheel gymnastics.

We propose a framework to analyze the relationship between the movement features of a wheel gymnast around the mounting phase of Unit 2 of the vault event and execution (E-score) deductions from a machine-learning perspective. We first developed an automation system from a video of a wheel gymnast performing a tuck-front somersault to extract the four frames highlighting its Unit 2 performance of the vault event, such as take-off, pike-mount, the starting point of time on the wheel, and final position before the thrust. We implemented this automation using recurrent all-pairs field transforms (RAFT) and XMem, i.e., deep network architectures respectively for optical flow estimation and video object segmentation. We then used a markerless pose-estimation system called OpenPose to acquire the coordinates of the gymnast's body joints, such as shoulders, hips, and knees then calculate the joint angles at the extracted video frames. Finally, we constructed a regression model to estimate the E-score deductions during Unit 2 on the basis of the joint angles using an ensemble learning algorithm called Random Forests, with which we could automatically select a small number of features with the nonzero values of feature importances. By applying our framework of markerless motion analysis to videos of male wheel gymnasts performing the vault, we achieved precise estimation of the E-score deductions during Unit 2 with a determination coefficient of 0.79. We found the two movement features of particular importance for them to avoid significant deductions: time on the wheel and angles of knees at the pike-mount position. The selected features well reflected the maturity of the gymnast's skills related to the motions of riding the wheel, easily noticeable to the judges, and their branching conditions were almost consistent with the general vault regulations.