Pose Detection Research Articles

Tomato 3D pose detection algorithm based on keypoint detection and point cloud processing

Tomatoes are widely grown all over the world and are one of the favourite vegetables of humanity. Tomato harvesting requires a lot of labor. With the aging population and the increasing demand for tomatoes, it is imminent to develop tomato picking robots. In the picking environment with a complex background, a 3D pose of the target is essential. It can provide guidance for robotic arm attitude and obstacle avoidance during picking. In this paper, a tomato pose detection algorithm (TPD) is proposed for 3D pose detection of a single fruit of clustered tomato. The TPD algorithm is divided into two modules: YOLO-lmk model and the point cloud processing module. By analyzing the network structure, optimizing parameters, optimizing loss function, adding attention mechanism and adding keypoint prediction, a YOLO-lmk model with YOLO v5s as the core is proposed to realize tomato bounding box and keypoint detection. The point cloud processing module is composed of point cloud segmentation, voxel downsampling, removing outliers, Euclidean color clustering, RANSAC sphere fitting, and keypoint index. As the experimental results show, YOLO-lmk model bounding box mAP is 92.9%, dlmk is 7.9, FLoating point Operations (FLOPs) is 16.6B, and speed is 0.062 s/sheet. The dlmk represents the Euclidean distance between true keypoints and predicted keypoints. Compared with YOLO v5s, mAP is increased 1.8%, and FLOPs is increased only 0.1B. The point cloud processing module only takes 0.028 s to complete a tomato 3D pose detection, which is fast. The accuracy of the TPD algorithm in detecting tomatoes is 93.4%, and the time cost is 0.09 s for detecting one tomato. The TPD algorithm can provide a theoretical basis for tomato 3D pose detection, and provide a reference for other fruits (pears, citrus, apples) 3D pose detection.

YOGA POSE DETECTION USING OPENPOSE AND MEDIAPIPE

An abstract i.e. idea behind this yoga pose detection project using deep learning or neural network learning is that yoga popularity is increasing day by day because of its benefits. Doing yoga helps us physically, mentally as well as spiritually. Because of this many people nowadays are doing it regularly. The main idea of this project is to help the people to recognize which yoga pose they are doing with the help of this detection technique. Yoga which involves 8 rungs and limbs of it, which includes Yama, Niyama, Asana, Pranayama, Dharana, Dhyana and Samadhi. To easily help people understand which pose they are performing via images, video recording by classifying it, we are implementing this project because of this people will incline towards doing more as they will get help to identify which pose they are doing very easily. Key Words: CNN, Deep learning, Feature Extraction,Yoga Pose Detection, openPose , mediaPipe

Smart Yoga Assistant: SVM-based Real-time Pose Detection and Correction System

SVM-based Real-time Pose Detection and Correction System refer to a computer system that uses machine learning techniques to detect and correct a person's yoga pose in real-time. This system can act as a virtual yoga assistant, helping people improve their yoga practice by providing immediate feedback on their form and helping to prevent injury. This paper presents a yoga tracker and correction system that uses computer vision and machine learning algorithms to track and correct yoga poses. The system comprises a camera and a computer vision module that captures images of the yoga practitioner and identifies the poses being performed. The machine learning module analyzes the images to provide feedback on the quality of the poses and recommends corrections to improve form and prevent injuries. This paper proposed a customized support vector machine (SVM) based real-time pose detection and correction system that suggests yoga practices based on specific health conditions or diseases. Paper aims to provide a reliable and accessible resource for individuals seeking to use yoga as a complementary approach to managing their health conditions. The system also includes a practitioner’s interface that enables practitioners to receive personalized recommendations for their yoga practice. The system is developed using Python and several open-source libraries, and was tested on a dataset of yoga poses. The hyper parameter gamma tuned to optimize the classification accuracy on our dataset produced 87% which is better than other approaches. The experiment results demonstrate the effectiveness of the system in tracking and correcting yoga poses, and its potential to enhance the quality of yoga practice.

Method for detecting pipeline spatial attitude using point cloud alignment

This paper introduces a new method for detecting the spatial attitude of pipeline welding based on point cloud registration. The method aligns the measured point cloud of the pipeline with the CAD model point cloud using point cloud registration and guides the pipeline adjustment accordingly. This automated measurement method has resolved the inefficiencies and potential for rework that can arise from manual instrument operation during pipeline pose detection, and has subsequently improved detection efficiency and accuracy. In addition, we propose a coarse registration algorithm based on the axial line of pipeline point clouds to help facilitate the alignment process. We conducted simulation experiments using 3D CAD models, which demonstrated that our coarse registration algorithm outperformed the other three compared methods in terms of accuracy. The simulation experiments also proved that the absolute value of the overall method's relative error was less than 0.1%.

Yoga Pose Detection

Abstract: This paper describes a real-time yoga pose detection system that can accurately classify and detect yoga poses in images using Convolutional Neural Networks (CNNs) and OpenPose. By using OpenPose, the system generates a 3D joint map of the person's body, which is then used as input for linear regression to detect the individual yoga pose. The system is suitable for real-time applications, and is expected to be used in fitness centers, yoga studios, and even for personal use. Additionally, the system can also be used to track the progress of yoga practitioners, allowing them to analyze their performance and improve their practice. Furthermore, the proposed system is expected to benefit the yoga industry by providing a low- cost, efficient, and accurate means to detect poses.

Monocular 3D Human Pose Markerless Systems for Gait Assessment.

Gait analysis plays an important role in the fields of healthcare and sports sciences. Conventional gait analysis relies on costly equipment such as optical motion capture cameras and wearable sensors, some of which require trained assessors for data collection and processing. With the recent developments in computer vision and deep neural networks, using monocular RGB cameras for 3D human pose estimation has shown tremendous promise as a cost-effective and efficient solution for clinical gait analysis. In this paper, a markerless human pose technique is developed using motion captured by a consumer monocular camera (800 × 600 pixels and 30 FPS) for clinical gait analysis. The experimental results have shown that the proposed post-processing algorithm significantly improved the original human pose detection model (BlazePose)'s prediction performance compared to the gold-standard gait signals by 10.7% using the MoVi dataset. In addition, the predicted T2 score has an excellent correlation with ground truth (r = 0.99 and y = 0.94x + 0.01 regression line), which supports that our approach can be a potential alternative to the conventional marker-based solution to assist the clinical gait assessment.

TPMv2: An end-to-end tomato pose method based on 3D key points detection

The automatic harvesting of tomatoes has been achieved for many years in the laboratory. The new research topic is harvesting the tomato more flexibly and nondestructively at any tomato bunch pose according to the agronomic demands. Although the tomato pose can be predicted by keypoints detection, the poor data quality of commercial RGBD cameras, occlusion between plant organs, various tomato poses, and unstructured working environments pose some challenges to the tomato bunch pose detection. Therefore, our research proposed an improved version of the Tomato Pose Method (TPM), namely TPMv2, which is a two-stage end-to-end multi-task network. This network provides comprehensive information on the tomato bunch, including the positions and poses of the stem, peduncle, and fruits, by predicting the two-dimensional bounding box (2D BBox), three-dimensional bounding box (3D BBox), two-dimensional key point (2D Kpt), and three-dimensional key point (3D Kpt). Aiming at the problems of occlusion and poor-quality point cloud, this paper specially designs a key point network (KPN) for tomatoes, where a keypoints processing pipeline was innovatively proposed, improving the accuracy of key point positioning and reducing abnormal prediction effectively. TPMv2 makes it possible to detect tomato bunch pose precisely with an economical camera, avoiding dangerous situations caused by abnormal prediction. The precision of 2D BBox and 3D BBox reached 0.9372 and 0.8700, and the Percentage of correct Keypoints (PCK) of 2D Kpt and 3D Kpt reached 0.8882 and 0.7836. About 78.36 % of 3D Kpts' positioning errors are less than 20 mm, sufficient to describe a correct pose trend based on the 3D Kpt, benefiting the manipulator to plan a more reasonable trajectory for non-destructive harvesting.

Live Yoga Pose Classification using Image Processing and Machine Learning

An approach to accurately recognize various Yoga Pose Assessment using deep learning algorithms has been presented in this work. In this system, we propose a Yoga pose assessment method using pose detection to help the self-learning of Yoga. The system first detects a yoga pose using multi parts detection only with PC camera. In this system, we also propose an improved algorithm to calculate scores that can be applied to all poses. Our application is evaluated on different Yoga poses under different scenes, and its robustness is also. A hybrid Machine learning model is proposed using linear Regression for Yoga recognition on real-time videos, where Linear regression is used to extract features from key-points of each frame obtained from Open Pose.

Effectual pre-processing with quantization error elimination in pose detector with the aid of image-guided progressive graph convolution network (IGP-GCN) for multi-person pose estimation

Multi-person pose estimation (MPE) remains a significant and intricate issue in computer vision. This is considered the human skeleton joint identification issue and resolved by the joint heat map regression network lately. Learning robust and discriminative feature maps is essential for attaining precise pose estimation. Even though the present methodologies established vital progression via feature map’s interlayer fusion and intralevel fusion, some studies show consideration for the combination of these two methodologies. This study focuses upon three phases of pre-processing stages like occlusion elimination, suppression strategy, and heat map methodology to lessen noise within the database. Subsequent to pre-processing errors will be eliminated by employing the quantization phase by embracing the pose detector. Lastly, Image-Guided Progressive Graph Convolution Network (IGP-GCN) has been built for MPE. This IGP-GCN consistently learns rich fundamental spatial information by merging features inside the layers. In order to enhance high-level semantic information and reuse low-level spatial information for correct keypoint representation, this also provides hierarchical connections across feature maps of the same resolution for interlayer fusion. Furthermore, a missing connection between the output high level information and low-level information was noticed. For resolving the issue, the effectual shuffled attention mechanism has been proffered. This shuffle intends to support the cross-channel data interchange between pyramid feature maps, whereas attention creates a trade-off between the high level and low-level representations of output features. This proffered methodology can be called Occlusion Removed_Image Guided Progressive Graph Convolution Network (OccRem_IGP-GCN), and, thus, this can be correlated with the other advanced methodologies. The experimental outcomes exhibit that the OccRem_IGP-GCN methodology attains 98% of accuracy, 93% of sensitivity, 92% of specificity, 88% of f1-score, 42% of relative absolute error, and 30% of mean absolute error.

A Quantifiable Telemedicine Approach to the Myasthenia Gravis Neurological Exam (P7-8.008)

<h3>Objective:</h3> Here we assessed whether quantification of videotaped assessment of the MG-CE could be done using novel image processing techniques. <h3>Background:</h3> The Myasthenia Gravis (MG) Core Exam (MG-CE) was developed to assist neurologists during telemedicine examination for patients with MG and consists of eight core elements that can be performed over video conferencing. However, much like in-person exams, they remain subjective to the interpretation and experience of the examiner. <h3>Design/Methods:</h3> We utilized video recordings of the ADAPT teleMG study (NINDS U54 NS115054) performed at George Washington University. Seven subjects underwent video examinations on two occasions 2–3 days apart and had a standardized examination recorded and videos analyzed using image-processing techniques. <h3>Results:</h3> We used a combination of artificial intelligence and signal processing algorithms to compute automatically the main geometric metrics: (i) eyelid drop, (ii) misalignment of eyes that can lead to double vision, (iii) cheek deformation in the cheek puff and tongue to cheek test, variation of extended arm position and sit to stand trajectory and velocity during the corresponding physical tests. These metrics provide reproductible and unbiased scores for each of the component of the MG-CE. We obtained in addition the time dependency of these muscle weakness metrics, which is a new outcome of the digitalization. <h3>Conclusions:</h3> With very promising results on eye tracking and pose detection, our method can provide objective and reproductible assessments. However, some exercises’ protocol are not fitting the robustness of our new metrics and should be revisited. As a matter of fact, the time dynamic component of muscle weakness seems an important factor in the evaluation of the symptoms. We envision that our telehealth approach can be applied to other neuromuscular diseases and will be providing quantitative metrics that goes beyond in an in-person visit. <b>Disclosure:</b> Prof. Garbey has stock in Care Constitution, LLC. Prof. Garbey has a non-compensated relationship as a CEO with Care Constitution, LLC that is relevant to AAN interests or activities. Dr. Joerger has nothing to disclose. Mr. Lesport has nothing to disclose. Helen Girma has nothing to disclose. Ms. McNett has nothing to disclose. Dr. Abu-Rub has nothing to disclose. The institution of Dr. Kaminski has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Roche. Dr. Kaminski has received personal compensation in the range of $500-$4,999 for serving as a Consultant for UCB. Dr. Kaminski has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Cabaletta Bio. Dr. Kaminski has stock in ARC Biotechnology. The institution of Dr. Kaminski has received research support from National Institutes of Health. Dr. Kaminski has received publishing royalties from a publication relating to health care.

Yoga AI Assistant with ReactJS

Yoga is beneficial for maintaining physical and mental fitness. And these days, technology is helping yoga find a new companion as online yoga becomes more and more common, making it easy to access classes and instructions. TensorFlow’s pose net, a tool for pose detection, is used by the AI yoga web app. Pre-trained models for identifying human poses are immediately accessible in the browser using posenet.js, a toolkit for estimating yoga poses based on body count rate determined by the pose. A person can receive feedback on the accuracy of their pose and keep track of their fitness goal in the application using real-time camera data.

Pose detection using deep learning for gait assessment

This approach of gait analysis helps in solving the problems associated with the health of aged people by concentrating on the frailty and senility syndromes that affect elderly individuals. Deterioration of cognitive and motor abilities is a common result of ageing and has an effect on elderly people's quality of life. Some studies have connected these changes in gait patterns to the decline in cognitive and motor function. As a result, gait analysis is a useful tool for diagnosing senility and frailty diseases. By transferring the pose estimation data to a correct statistical analysis, gait analysis can be carried out utilizing machine learning and computer vision approach, allowing physiotherapists to analyse and treat patients accordingly. The main problem observed is that physiotherapists will analyse stroke patients' walking patterns on their own. As a result, this approach solves the problem by providing an accurate result of the observation. As everything is computerised, proper treatment can be provided with accurate data. A physiotherapist could also use machine learning classification techniques to analyse and classify the observed disorder and correct it. Our study mainly focuses on using machine learning approach instead of traditional sensor based or consultation just by vision. This will serve as a newer and better proposition of the same.

Pixel-Coordinate-Induced Human Pose High-Precision Estimation Method

Accurately estimating human pose is crucial for providing feedback during exercises or musical performances, but the complex and flexible nature of human joints makes it challenging. Additionally, traditional methods often neglect pixel coordinates, which are naturally present in high-resolution images of the human body. To address this issue, we propose a novel human pose estimation method that directly incorporates pixel coordinates. Our method adds a coordinate channel to the convolution process and embeds pixel coordinates into the feature map, while also using coordinate attention to capture position- and structure-sensitive features. We further reduce the network parameters and computational cost by using small-scale convolution kernels and a smooth activation function in residual blocks. We evaluate our model on the MPII Human Pose and COCO Keypoint Detection datasets and demonstrate improved accuracy, highlighting the importance of directly incorporating coordinate location information in position-sensitive tasks.

Body Pose Estimation using Deep Learning

Abstract: Healthcare, sports analysis, gaming, and entertain- ment are just some of the many fields that could benefit from solving the challenging issue of real-time human pose detection and recognition in computer vision. Capturing human motion, analysing physical exercise, and giving feedback on performance can all benefit from reliable detection and recognition of body poses. The recent progress in deep learning has made it possible to create real-time systems that can accurately and quickly recognise and identify human poses.

Grasping Complex-Shaped and Thin Objects Using a Generative Grasping Convolutional Neural Network

Vision-based pose detection and grasping complex-shaped and thin objects are challenging tasks. We propose an architecture that integrates the Generative Grasping Convolutional Neural Network (GG-CNN) with depth recognition to identify a suitable grasp pose. First, we construct a training dataset with data augmentation to train a GG-CNN with only RGB images. Then, we extract a segment of the tool using a color segmentation method and use it to calculate an average depth. Additionally, we apply and evaluate different encoder–decoder models with a GG-CNN structure using the Intersection Over Union (IOU). Finally, we validate the proposed architecture by performing real-world grasping and pick-and-place experiments. Our framework achieves a success rate of over 85.6% for picking and placing seen surgical tools and 90% for unseen surgical tools. We collected a dataset of surgical tools and validated their pick and place with different GG-CNN architectures. In the future, we aim to expand the dataset of surgical tools and improve the accuracy of the GG-CNN.

Open-Set Object Detection Using Classification-Free Object Proposal and Instance-Level Contrastive Learning

Detecting both known and unknown objects is a fundamental skill for robot manipulation in unstructured environments. Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. In this letter, we present Openset RCNN to address the challenging OSOD. To disambiguate unknown objects and background in the first subtask, we propose to use classification-free region proposal network (CF-RPN) which estimates the objectness score of each region purely using cues from object's location and shape preventing overfitting to the training categories. To identify unknown objects in the second subtask, we propose to represent them using the complementary region of known categories in a latent space which is accomplished by a prototype learning network (PLN). PLN performs instance-level contrastive learning to encode proposals to a latent space and builds a compact region centering with a prototype for each known category. Further, we note that the detection performance of unknown objects can not be unbiasedly evaluated on the situation that commonly used object detection datasets are not fully annotated. Thus, a new benchmark is introduced by reorganizing GraspNet-1billion, a robotic grasp pose detection dataset with complete annotation. Extensive experiments demonstrate the merits of our method. We finally show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments.

Discriminative Active Learning for Robotic Grasping in Cluttered Scene

Robotic grasping is a challenging task due to the diversity of object shapes. A sufficiently labeled dataset is essential for the grasp pose detection methods based on deep learning. However, data annotation is a costly procedure. Active learning aims to mitigate the greedy need for massive labeled data. In this work, we propose a Discriminative Active Learning (DAL) framework for robotic grasping algorithms. DAL is an effective strategy that utilizes a shared encoder to derive latent features from both labeled data and unlabeled data. A discriminator is established to estimate the informativeness of each unlabeled data sample and decide whether they should be annotated for the next epoch. Moreover, an annotation interface is also developed to annotate the chosen data. We evaluate DAL with real-world grasp datasets and show superior performance, especially when the amount of labeled data is little. Considering annotation noise, we perform an experiment on a noisy dataset and demonstrate that our proposed framework is stable to noisy annotation. Besides, we train a model with about 60% data selected by DAL of the whole dataset and it can still handle a real-world grasp detection task in cluttered scene on a real robot.

AI Application for Body Stiffness in Parkinsons Disease

Abstract: National projections indicate that the number of people over 60 years old is expected to increase, bringing with it an increase in the number of people affected by Parkinson's Disease (PD). This makes PD an important public health problem. Therefore, the development of effective approaches for intervention in people with Parkinson's disease needs to be more thoroughly investigated. End-to-end application will make exercising at home more convenient for patients. We can project prescribed exercises and determine whether the patient is completing them correctly. The Flutter app enables patients to register themselves, communicate with specialists over the app, and make virtual appointments. The pose detection library will help us detect angles of the exercises, and determine whether patients are performing them correctly

Empirical Mode Decomposition and Hilbert Spectrum for Abnormality Detection in Normal and Abnormal Walking Transitions.

Sensor-based human activity recognition (HAR) is a method for observing a person's activity in an environment. With this method, it is possible to monitor remotely. HAR can analyze a person's gait, whether normal or abnormal. Some of its applications may use several sensors mounted on the body, but this method tends to be complex and inconvenient. One alternative to wearable sensors is using video. One of the most commonly used HAR platforms is PoseNET. PoseNET is a sophisticated platform that can detect the skeleton and joints of the body, which are then known as joints. However, a method is still needed to process the raw data from PoseNET to detect subject activity. Therefore, this research proposes a way to detect abnormalities in gait using empirical mode decomposition and the Hilbert spectrum and transforming keys-joints, and skeletons from vision-based pose detection into the angular displacement of walking gait patterns (signals). Joint change information is extracted using the Hilbert Huang Transform to study how the subject behaves in the turning position. Furthermore, it is determined whether the transition goes from normal to abnormal subjects by calculating the energy in the time-frequency domain signal. The test results show that during the transition period, the energy of the gait signal tends to be higher than during the walking period.

Weakly Supervised 2D Pose Adaptation and Body Part Segmentation for Concealed Object Detection.

Weakly supervised pose estimation can be used to assist unsupervised body part segmentation and concealed item detection. The accuracy of pose estimation is essential for precise body part segmentation and accurate concealed item detection. In this paper, we show how poses obtained from an RGB pretrained 2D pose detector can be modified for the backscatter image domain. The 2D poses are refined using RANSAC bundle adjustment to minimize the projection loss in 3D. Furthermore, we show how 2D poses can be optimized using a newly proposed 3D-to-2D pose correction network weakly supervised with pose prior regularizers and multi-view pose and posture consistency losses. The optimized 2D poses are used to segment human body parts. We then train a body-part-aware anomaly detection network to detect foreign (concealed threat) objects on segmented body parts. Our work is applied to the TSA passenger screening dataset containing millimeter wave scan images of airport travelers annotated with only binary labels that indicate whether a foreign object is concealed on a body part. Our proposed approach significantly improves the detection accuracy of TSA 2D backscatter images in existing works with a state-of-the-art performance of 97% F1-score, 0.0559 log-loss on the TSA-PSD test-set, and a 74% reduction in 2D pose error.