Pose Detection Research Articles

Geometric error self-calibration method of five-axis dispensing machine based on the product of exponentials formula

Geometric error identification and kinematic calibration are common and effective methods for improving the machining accuracy of five-axis machines. Compared with the traditional measurement methods characterized by complicated inspection processes and high cost, this paper proposes a self-calibration method for a five-axis dispensing machine based on monocular vision and the product of the exponentials formula. The proposed method can automatically identify geometric errors and complete kinematic calibration, even for simultaneous five-axis motion. First, a ChArUco marker-based pose detection system is designed to achieve a wide range of calibration space for a five-axis dispensing machine. Based on this, the proposed sub-pixel checkpoint detection framework has higher pose detection performance accuracy at different depths of the field. Then, a general kinematic error model of five-axis machines is established using screw theory to avoid singularities. Moreover, the identification method of its geometric error is proposed for an RRTTT-type five-axis dispensing machine. Finally, an arc-shaped dispensing trajectory for the 3D glasses' inner wall is used for experimental validation. The experimental results show that the average orientation quantization error decreases from 0.0112 rad to 0.0039 rad. In contrast, the average position error decreases from 0.1627 mm to 0.0473 mm before and after the self-calibration, respectively. Thus, the convenience, effectiveness, and accuracy of the self-calibration method are validated on this five-axis dispensing machine.

Efficient Sampling of Two-Stage Multi-Person Pose Estimation and Tracking from Spatiotemporal

Tracking the articulated poses of multiple individuals in complex videos is a highly challenging task due to a variety of factors that compromise the accuracy of estimation and tracking. Existing frameworks often rely on intricate propagation strategies and extensive exchange of flow data between video frames. In this context, we propose a spatiotemporal sampling framework that addresses the degradation of frames at the feature level, offering a simple yet effective network block. Our spatiotemporal sampling mechanism empowers the framework to extract meaningful features from neighboring video frames, thereby optimizing the accuracy of pose detection in the current frame. This approach results in significant improvements in running latency. When evaluated on the COCO dataset and the mixed dataset, our approach outperforms other methods in terms of average precision (AP), recall rate (AR), and acceleration ratio. Specifically, we achieve a 3.7% increase in AP, a 1.77% increase in AR, and a speedup of 1.51 times compared to mainstream state-of-the-art (SOTA) methods. Furthermore, when evaluated on the PoseTrack2018 dataset, our approach demonstrates superior accuracy in multi-object tracking, as measured by the multi-object tracking accuracy (MOTA) metric. Our method achieves an impressive 11.7% increase in MOTA compared to the prevailing SOTA methods.

Vision-based food handling system for high-resemblance random food items

AbstractThe rise in the number of automated robotic kitchens accelerated the need for advanced food handling system, emphasizing food analysis including ingredient classification pose recognition and assembling strategy. Selecting the optimal piece from a pile of similarly shaped food items is a challenge to automated meal assembling system. To address this, we present a constructive assembling algorithm, introducing a unique approach for food pose detection–Fast Image to Pose Detection (FI2PD), and a closed-loop packing strategy. Powered by a convolutional neural network (CNN) and a pose retrieval model, FI2PD is adept at constructing a 6D pose from only RGB images. The method employs a coarse-to-fine approach, leveraging the CNN to pinpoint object orientation and position, alongside a pose retrieval process for target selection and 6D pose derivation. Our closed-loop packing strategy, aided by the Item Arrangement Verifier, ensures precise arrangement and system robustness. Additionally, we introduce our FdIngred328 dataset of nine food categories ranging from fake foods to real foods, and the automatically generated data based on synthetic techniques. The performance of our method for object recognition and pose detection has been demonstrated to achieve a success rate of 97.9%. Impressively, the integration of a closed-loop strategy into our meal-assembly process resulted in a notable success rate of 90%, outperforming the results of systems lacking the closed-loop mechanism.

A Robotic Arm Visual Grasp Detection Algorithm Combining 2D Images and 3D Point Clouds

Robot grasping detection methods are categorized into two-dimension (2D) and three-dimension (3D) approaches. In 2D grasp detection, gripper pose prediction occurs directly on RedGreenBlue (RGB) images, limiting grasp direction. Conversely, 3D grasp detection predicts gripper pose using 3D point clouds, allowing greater grasp flexibility. However, the data volume of 3D point clouds hampers real-time detection. To address this, this paper proposes a novel grasping detection algorithm that combines 2D images and 3D point clouds. Initially, a Single Shot MultiBox Detector (SSD) network generates 2D prediction boxes on RGB images. Through enhancements to the prior box scaling, the accuracy of bounding the target object is improved. Subsequently, 2D boxes are transformed into 3D frustums, and extraneous data points are removed. By utilizing Random Sampling Consistent Segmentation (RANSAC) and Euclidean Clustering Segmentation Algorithm (ECSA), the target point clouds are isolated, and subsequently, the spatial pose of the target is represented using an Oriented Bounding Box (OBB). Processed point clouds enter an enhanced PointNet Grasp Pose Detection (PointNetGPD) algorithm, In contrast to the original approach involving extensive random sampling of grasp candidates, the enhanced PointNetGPD method enables the selective sampling of grasp candidates by incorporating pose constraints between the target and gripper. Following this, the generated grasp candidates are subjected to evaluation through a scoring process conducted by an assessment network. Ultimately, the robotic arm is guided to perform the grasp associated with the highest score. In the experimental phase, the proposed algorithm demonstrated a high success rate in capturing multiple targets, along with a reduced grasping time. These results underscore the algorithm's superior grasping quality and enhanced real-time performance when compared to similar algorithms.

Review on Development of Automatic Docking System for Weapons and Cabins

Background: In the manufacturing setup process of weapons, aviation, and aerospace, docking assembly is a key component in determining manufacturing efficiency and quality. The traditional docking method uses manual alignment, which depends on the experience and operation level of the workers, so the docking accuracy is low and the installation speed is slow. In response to this problem, how to achieve automatic docking of weapons and cabins has become a key technology. Objective: The objective of this study is to provide an overview of recent key technologies for automatic docking systems and introduce their respective classification, characteristics, and development. Methods: This paper reviews the patents and products related to the key technologies of automatic docking systems. The structural features, methodological principles, control characteristics and applications of the patents and products are summarized. Results: The key technologies related to different aspects of an automatic docking system are compared and analyzed, and the structural features, and methodological principles are also summarized. The problems of the current key technologies are analyzed, and the current status of research and future directions of the automatic docking system are discussed. Conclusion: According to the key technologies related to different aspects of the automatic docking system, they can be divided into motion mechanism, pose detection technology, and computer control technology. The accuracy, compliance, and flexibility of the automatic docking system can be improved by choosing the proper key technology. Further improvements are needed in terms of fast performance, robustness and authenticity, and more patents on key technologies of automatic docking systems should be invented.

FastGNet: an efficient 6-DOF grasp detection method with multi-attention mechanisms and point transformer network

A pivotal technology for autonomous robot grasping is efficient and accurate grasp pose detection, which enables robotic arms to grasp objects in cluttered environments without human intervention. However, most existing methods rely on PointNet or convolutional neural network as backbones for grasp pose prediction, which may lead to unnecessary computational overhead on invalid grasp points or background information. Consequently, performing efficient grasp pose detection for graspable points in complex scenes becomes a challenge. In this paper, we propose FastGNet, an end-to-end model that combines multiple attention mechanisms with the transformer architecture to generate 6-DOF grasp poses efficiently. Our approach involves a novel sparse point cloud voxelization technique, preserving the complete mapping between points and voxels while generating positional embeddings for the transformer network. By integrating unsupervised and supervised attention mechanisms into the grasp model, our method significantly improves the performance of focusing on graspable target points in complex scenes. The effectiveness of FastGNet is validated on the large-scale GraspNet-1Billion dataset. Our approach outperforms previous methods and achieves relatively fast inference times, highlighting its potential to advance autonomous robot grasping capabilities.

Pengembangan Aplikasi Pose Detection untuk Asesmen Kemajuan Fisioterapi Pasien Pasca Stroke dari Jarak Jauh

Assessment has an important role in determining the diagnosis and subsequent treatment plan. In an effort to increase access and effectiveness of rehabilitation, this research aims to develop a mobile application that is able to report the results of post-stroke patient pose assessment remotely. Telemedicine approaches in post-stroke rehabilitation have become increasingly popular, allowing patients to access rehabilitation services remotely. This is especially beneficial for patients who live in remote areas or have limited mobility. Telemedicine also allows for real-time patient monitoring, allowing adjustments to rehabilitation plans as needed. The mobile app is designed to provide easy access to rehabilitation programs that can be tailored to individual patient needs. In addition to making access easier, this application is equipped with a monitoring feature that allows health professionals to follow patient progress in detail. Data collected from patients' daily exercise and activities provides valuable insight into their progress, which can be used in tailoring rehabilitation plans in real-time. The development of this mobile application technology has great potential to improve rehabilitation outcomes for post-stroke patients. Testing by three experts with two experts as healthy patients and stroke patients, as well as one patient who acts as a medical personel to monitor, shows that from the graph, healthy patients tend to be consistent. On the other hand, post-stroke patients tend to be inconsistent. These results indicate that this application is effective for identifying patient movements during the rehabilitation process. Although there are several obstacles, such as delays in predictions on some devices, this application has great potential to improve the quality of life of post-stroke patients. Thus, the development of a pose detection application for remotely assessing the progress of physiotherapy in post-stroke patients has great potential in improving rehabilitation outcomes. The app facilitates patient access to appropriate, personalized and effective care, while providing medical personnel with objective and accurate data for monitoring and adjusting rehabilitation plans. This is a significant step in advancing the care of post-stroke patients.

Comprehensive visual information acquisition for tomato picking robot based on multitask convolutional neural network

The tomato picking robot's vision system faces two difficult tasks: precise tomato pose acquisition and stem location. Tomato pose and stem location can help determine the end effector pose and achieve collision-free picking. To realise efficient crop picking, the tasks of target location, pose detection, and obstacle semantic segmentation should be completed in one model to obtain comprehensive visual information. Therefore, the multitask convolutional neural network YOLO-MCNN is proposed, a new method to complete the above tasks in one model. By fusing multi-scale features and determining the optimal locations for the semantic segmentation branch, four strategies are proposed for enhancing the segmentation ability. The experiment results show that fusing the semantic segmentation branch with the second layer of shallow feature maps and placing the branch after the 17th layer can result in the best segmentation performance. Fusing shallow feature maps improves small target detection while merging multi-scale feature maps enhances semantic segmentation performance. Moreover, ablation experiments are conducted to understand the influence between multitask convolutional and single task networks. It proves that running multiple tasks on the same backbone network does not affect their performance. The YOLO-MCNN's target detection performance F1 is 87.8%, the semantic segmentation performance mIoU is 74.8%, the keypoint detection performance dlmk is 6.95 pixels, the network size is 15.2 MB, and the inference speed is 19.9ms. Compared with other target detection and semantic segmentation networks, it shows that the comprehensive performance of the YOLO-MCNN is the best. The method provides theoretical foundation for constructing multitask convolutional neural networks.

Comparison of Gradient Boosting and Random Forest Models in the Detection System of Rakaat during Prayer

Abstract. Errors in the execution of prayer among Muslims can occur due to a lack of profound understanding of the prayer procedure. This research aims to compare two machine learning models, Random Forest and Gradient Boosting, in classifying prayer movements, subsequently extending to calculate the number of prayer cycles (rakaat). A total of 7220 manually gathered data based on 33 landmark coordinates using Mediapipe Pose Detection were employed. The research findings reveal that the Random Forest model with a 70:30 ratio achieves 99.9% accuracy, precision, and recall, with the fastest training time being 3.8 seconds. Both models exhibit testing results close to 100%, but the Gradient Boosting model faces challenges in classifying specific movements. On the other hand, Random Forest successfully overcomes thesechallenges, enabling accurate prayer cycle calculations. The findings can contribute to the development of tools supporting Muslims in correct prayer execution, positively impacting religious and well-being aspects.

Surface and underwater human pose recognition based on temporal 3D point cloud deep learning

Airborne surface and underwater human pose recognition are crucial for various safety and surveillance applications, including the detection of individuals in distress or drowning situations. However, airborne optical cameras struggle to achieve simultaneous imaging of the surface and underwater because of limitations imposed by visible-light wavelengths. To address this problem, this study proposes the use of light detection and ranging (LiDAR) to simultaneously detect humans on the surface and underwater, whereby human poses are recognized using a neural network designed for irregular data. First, a temporal point-cloud dataset was constructed for surface and underwater human pose recognition to enhance the recognition of comparable movements. Subsequently, radius outlier removal (ROR) and statistical outlier removal (SOR) were employed to alleviate the impact of noise and outliers in the constructed dataset. Finally, different combinations of secondary sampling methods and sample sizes were tested to improve recognition accuracy using PointNet++. The experimental results show that the highest recognition accuracy reached 97.5012%, demonstrating the effectiveness of the proposed human pose detection and recognition method.

Contact-based pose detection method for small components to optimise the digital twin-driven robotic assembly process

The robotic assembly has become pivotal in manufacturing, demanding precise pose detection of assembly components for efficient operations. This paper presents a contact-based pose detection method tailored for digital twin-based optimisation for the robotic assembly process, focusing on small component assembly challenges. The proposed technique achieves robust pose estimation by leveraging a strain gauge load cell-based calibration method. Experimental validation demonstrates close alignment between induced and measured errors, showcasing the efficacy of the method in mitigating assembly challenges. Despite minor deviations, the approach outperforms traditional vision-based methods, promising enhanced efficiency in robotic assembly tasks. Further refinement could bolster accuracy, fostering advanced robotic assembly capabilities.

Gesture-to-Text: A Real-Time Indian Sign Language Translator with Pose Estimation and LSTMs

In recent years, there have been notable advancements in technology, deep learning, and pose detection. One significant application of these advancements pertains to the real-time detection of sign language from video sources. The motivation behind this research stems from the pressing societal need to enhance the quality of life for individuals with speech impairments. Given the current prominence of online meetings, exacerbated by the COVID-19 pandemic, there is a growing need for systems that can provide individuals with speech impairments greater independence in communication, eliminating the requirement for a human translator. This research proposal advocates for a solution that leverages PoseNet algorithms for the extraction of key pose points, which are subsequently employed within LSTM models for the predictive modeling of sign language gestures. This research paper aims to make several notable contributions to the field of assistive technology and human-computer interaction. The achieved accuracy stands at an impressive 98%, underscoring the robustness and precision of our proposed system.

Fall Detection in Low-Illumination Environments From Far-Infrared Images Using Pose Detection and Dynamic Descriptors

Fall Detection in Low-Illumination Environments From Far-Infrared Images Using Pose Detection and Dynamic Descriptors

Energy-Efficient Implementation of YOLOv8, Instance Segmentation, and Pose Detection on RISC-V SoC

Energy-Efficient Implementation of YOLOv8, Instance Segmentation, and Pose Detection on RISC-V SoC

Interest Point Selection and Feature Extraction in Six-DoF Grasp Pose Detection

Interest Point Selection and Feature Extraction in Six-DoF Grasp Pose Detection

Implementation and Assessment of Convolutional Neural Networks for Non-Invasive Monitoring of Shark Physiology and BMI Calculation Using YOLOv8 Pose Detection

Implementation and Assessment of Convolutional Neural Networks for Non-Invasive Monitoring of Shark Physiology and BMI Calculation Using YOLOv8 Pose Detection

Personalized Gym Trainer Using Mediapipe

Abstract: With the growing demand in personalised fitness experiences, this paper takes a fresh look at home workouts by creating a Personalised Gym Trainer with the Mediapipe library. This sophisticated device combines pose detection technology with voice assistance to provide users with real-time feedback and personalised instruction while exercising. The system identifies various exercises accurately and leverages torso point detection for greater precision by leveraging the capabilities of the Mediapipe library and OpenCV for camera tasks. Individual Python modules for certain workouts such as pull-ups and bench press are written to support a flexible and scalable solution. Python, Git, GitHub, and Jenkins are among the tools and technologies used in the process. Furthermore, the Findpose library is used to calculate angles between torso locations, resulting in Providing a quantitative assessment of proper posture. The suggested Personalised Gym Trainer is a promising leap in home fitness solutions, integrating cutting-edge pose detection with voice assistance for an interactive and personalised workout experience.

Yoga Pose Detection for Human Rehabilitation Using Deep Leaning

Abstract: Yoga is an ancient science and discipline originated in India 5000 years ago. It is used to bring harmony to both body and mind with the help of asana, meditation and various other breathing techniques It bring peace to the mind. Due to increase of stress in the modern lifestyle, yoga has become popular throughout the world. There are various ways through which one can learn yoga. Yoga can be learnt by attending classes at a yoga centre or through home tutoring. It can also be self-learnt with the help of books and videos. Most people prefer self-learning but it is hard for them to find incorrect parts of their yoga poses by themselves. idea behind this yoga pose detection project using deep learning or neural network learning is that yoga popularity is increasing day by day because of its benefits. Doing yoga helps us physically, mentally as well as spiritually. Because of this many people nowadays are doing it regularly. The main idea of this project is to help the people to recognize which yoga pose they are doing with the help of this detection technique. Yoga which involves 8 rungs and limbs of it, which includes Yama, Niyama, Asana, Pranayama, Dharana, Dhyana and Samadhi. To easily help people understand which pose they are performing via images, video recording by classifying it, we are implementing this project because of this people will incline towards doing more as they will get help to identify which pose they are doing very easily.

Pose-Aided Video-Based Person Re-Identification via Recurrent Graph Convolutional Network

Existing methods for video-based person re-identification (ReID) mainly learn the appearance feature of a given pedestrian via a feature extractor and a feature aggregator. However, the appearance models would fail to learn a large inter-class variance when different pedestrians have similar appearances. Considering that different pedestrians have different walking postures and body proportions, we propose to learn the discriminative pose feature beyond the appearance feature for video retrieval. Specifically, we implement a two-branch architecture to separately learn the appearance feature and pose feature, and then concatenate them together for inference. To learn the pose feature, we first detect the pedestrian pose in each frame through an off-the-shelf pose detector, and construct a temporal graph using the pose sequence. We then exploit a recurrent graph convolutional network (RGCN) to learn the node embeddings of the temporal pose graph, which devises a global information propagation mechanism to simultaneously achieve the neighborhood aggregation of intra-frame nodes and message passing among inter-frame graphs. Finally, we propose a dual-attention method (DAM) consisting of node-attention and time-attention to obtain the temporal graph representation from the node embeddings, where the self-attention mechanism is employed to learn the importance of each node and each frame. We verify the proposed method on three video-based ReID datasets, i.e., Mars, DukeMTMC and iLIDS-VID, whose experimental results demonstrate that the learned pose feature can effectively improve the performance of existing appearance models.

Servo-controlled Semi-rigid Exoskeleton Designed for the Human Knee

Musculoskeletal injuries are occurring more and more frequently, with one of the main causes being the increasing number of new patients who sustain knee structure injuries. Currently, exoskeletons that assist the knee joint are either fully rigid or soft systems, which induce the risk of misalignments due to the complicated nature of the kinematics of the knee joint or require additional anchors or mechanical parts that may cause discomfort or force unnecessary weight upon the user. In this paper, a semi-rigid exoskeleton principle is introduced. This exoskeleton addresses the concerns of both rigid and soft exoskeletons while performing the same benefits by establishing a pre-determined route for the tendon. This exoskeleton can provide up to 37.26Nm assistive torque to the knee joint. The system’s effectiveness was validated through a series of experiments which included testing its range of motion, total tensile force, and accuracy and safety of the pose detection system. The prototype had successful results, and the overall system is efficient, flexible, and easy to use. Future work will focus more on a customizable system in which the users can determine the amount of assistive torque force they need to be applied to their knee joint depending on their current physical ability.