Pose Detection Research Articles

Virtual Yoga Assistant using Machine Learning and Artificial Intelligence

In recent years, yoga has become an integral part of life for people worldwide. This growing interest has created a demand for scientific analysis of yoga postures. Pose detection techniques offer a promising approach to identify and assist people in performing yoga poses with greater accuracy. However, posture recognition remains a challenging task due to the limited availability of datasets and the difficulty of real-time posture detection. To address this, a large dataset has been developed with over 5,500 images representing ten different yoga poses. A tf-pose estimation algorithm is used to create a skeletal overlay on each image in real time, drawing the human body’s skeleton to extract joint angles. These joint angles serve as features for training various machine learning models. The dataset is split, with 80% used for training and 20% for testing. This approach has been tested on multiple machine learning classification models, achieving an accuracy of 99.04% with a Random Forest Classifier.

Pasteables: A Flexible and Smart “Stick-and-Peel” Wearable Platform for Fitness and Athletics

Wearable technologies are becoming increasingly popular and transforming our fitness and sports industry. Monitoring health parameters and body activity can help achieve fitness goals, improve sports performance, and aid in physiotherapy. However, state-of-the-art wearable devices are often not flexible, expensive, hard to set up, or have privacy concerns. They are designed and optimized for a specific application with tight hardware-software integration and it is hard or impossible to re-purpose them for diverse use cases. In this paper, we propose a flexible, re-configurable human body movement and health monitoring platform, called "Pasteables". It is a stick-and-peel device (which acts like a band-aid) that attaches to the skin or clothing and can create an on-body network of smart wearables. Given an application, Pasteables can be easily adapted to its requirements while ensuring good performance and privacy. We present the overall architecture, system design steps, and the configuration process. We have designed a set of functional prototypes and configured them for performing a variety of camera-less pose and posture detection tasks, which are important for athletes, professional dancing, physiotherapy, and fitness workouts. We note that the Pasteables can perform pose/posture detection without external stationary reference at low cost and high accuracy.

A sensing system and solving method for dynamic detection of relative pose of hydraulic support group

The precise detection of the pose of hydraulic support groups is crucial for the construction of intelligent coal mines. Addressing current challenges in comprehensive position sensing of support groups, such as one-sided descriptions, missing information, and limited detection capabilities, this paper, based on the concept of digital twin, proposes a sensing system and solving method for dynamic detection of the relative pose of hydraulic support group. Initially, a relative pose detection model, based on the six-point localization principle, is proposed by combining existing sensor information. Then, a sensing system is designed, incorporating a photoelectric detection device for monitoring key position parameters and virtual sensors for determining pose information. Consequently, a method for dynamically deriving relative positions, driven by the fusion of real-virtual iterative solving, real-virtual error feedback, and iterative condition optimization, is established. Finally, a relative pose dynamic detection system for a hydraulic support prototype is constructed in a laboratory environment. Experimental results demonstrate that the sensor system and the relative position detection method designed in this paper achieve a relative position accuracy of over 95.32 % for the hydraulic support, with a dynamic reconstruction time of less than 0.2 s. This verifies the rationality of the sensor system and the feasibility and accuracy of the dynamic deduction method. To sum up, this work provides a theoretical foundation and technical support for the autonomous perception of hydraulic support group, autonomous frame adjustment between frames, and overall straightness adjustment. It also lays the groundwork for further research and the advancement of information-physical systems in the fully-mechanized mining face.

Augmented and Virtual Reality Based Segmentation Algorithm for Human Pose Detection in Wearable Cameras

Pose graph optimisation is a crucial method that helps reduce cumulative errors while estimating visual trajectories for wearable cameras. However, when the posture graph's size increases with each additional camera movement, the optimization's efficiency diminishes. In terms of ongoing sensitive applications, such as extended reality and computer-generated reality, direction assessment is a major test. This research proposes an incremental pose graph segmentation technique that accounts for camera orientation variations as a solution to this challenge. The computation only improves the cameras that have seen large direction changes by breaking the posture chart during these instances. As a result, pose graph optimisation is essentially slowed down and optimised more quickly. For every camera that hasn't been optimised using a pose graph, the algorithm employs the wearable cameras at the start and end of each camera's trajectory segment. The final camera in attendance is then determined by weighted average the various postures evaluated with these wearable cameras; this eliminates the need for lengthy nonlinear enhancement computations, reduces disturbance, and achieves excellent accuracy. Experiments on the EuRoC, TUM, and KITTI datasets demonstrate that pose graph optimisation scope is reduced while maintaining camera trajectories accuracy.

D3L-SLAM: A Comprehensive Hybrid Simultaneous Location and Mapping System with Deep Keypoint, Deep Depth, Deep Pose, and Line Detection

Robust localization and mapping are crucial for autonomous systems, but traditional handcrafted feature-based visual SLAM often struggles in challenging, textureless environments. Additionally, monocular SLAM lacks scale-aware depth perception, making accurate scene scale estimation difficult. To address these issues, we propose D3L-SLAM, a novel monocular SLAM system that integrates deep keypoints, deep depth estimates, deep pose priors, and a line detector. By leveraging deep keypoints, which are more resilient to lighting variations, our system improves the robustness of visual SLAM. We further enhance perception in low-texture areas by incorporating line features in the front-end and mitigate scale degradation with learned depth estimates. Additionally, point-line feature constraints optimize pose estimation and mapping through a tightly coupled point-line bundle adjustment (BA). The learned pose estimates refine the feature matching process during tracking, leading to more accurate localization and mapping. Experimental results on public and self-collected datasets show that D3L-SLAM significantly outperforms both traditional and learning-based visual SLAM methods in localization accuracy.

Visual-Inertial Fusion-Based Five-Degree-of-Freedom Motion Measurement System for Vessel-Mounted Cranes

Vessel-mounted cranes operate in complex marine environments, where precise measurement of cargo positions and attitudes is a key technological challenge to ensure operational stability and safety. This study introduces an integrated measurement system that combines vision and inertial sensing technologies, utilizing a stereo camera and two inertial measurement units (IMUs) to capture cargo motion in five degrees of freedom (DOF). By merging data from the stereo camera and IMUs, the system accurately determines the cargo’s position and attitude relative to the camera. The specific methodology is introduced as follows: First, the YOLO model is adopted to identify targets in the image and generate bounding boxes. Then, using the principle of binocular disparity, the depth within the bounding box is calculated to determine the target’s three-dimensional position in the camera coordinate system. Simultaneously, the IMU measures the attitude of the cargo, and a Kalman filter is applied to fuse the data from the two sensors. Experimental results indicate that the system’s measurement errors in the x, y, and z directions are less than 2.58%, 3.35%, and 3.37%, respectively, while errors in the roll and pitch directions are 3.87% and 5.02%. These results demonstrate that the designed measurement system effectively provides the necessary motion information in 5-DOF for vessel-mounted crane control, offering new approaches for pose detection of marine cranes and cargoes.

Influence of video instruction on soccer kick velocity in young players

Soccer is a sport that necessitates proficiency in motor control and cognitive abilities, with kicking serving as a fundamental action. Improving kicking performance is vital for player advancement, leading to the inquiry: “How can video instruction promptly optimize kicking technique in young players?”. This research aimed to investigate the immediate effects of an instructional video on the kinematic variables of kicking to induce an acute alteration in the movement pattern. A cohort of 26 male youths, aged between 10 and 15 years, partook in the study and were segregated into an instruction group (IG) and a control group (CG). The IG received exposure to a video containing specific kicking instructions, while the CG viewed a video lacking such guidance. Both groups were recorded performing penalty kicks before and after viewing the video. Kinematic analysis was carried out utilizing the OpenPose artificial intelligence neural network for pose detection and subsequent 3D reconstruction. The primary discovery was that the length of the last stride (LS) notably increased in the IG following the instructional video. This outcome indicates that video instructions hold promise in promptly enhancing particular aspects of kicking technique in young soccer players.

A novel methodology for analysing dental implant positional changes from virtual planning to placement without CBCT.

Introduction This paper introduces a novel methodology for analysing dental implant positioning in vivo, advancing beyond traditional methods that rely on post-placement cone beam computed tomography (CBCT) scans.Method The core of the methodology is comparing stereolithography (STL) files, representing pre-planned and actual post-placement implant positions. These STL files, exported from guided surgery planning and computer-aided design software, focus on clinically significant key points, like the apical and coronal midpoints. Additionally, the method uses pose detection, differing fundamentally from CBCT scan approximations.Discussion Relying on pose detection instead of scanner resolution, this method aligns with international standards and overcomes CBCT and intra-oral scanner limitations. It allows for a more precise and accurate assessment of implant positions, independent of scanner technology constraints. Further refinements include potential detailed reporting of apical deviations, enhancing implant placement accuracy.Conclusion This research has significant implications for dental implantology, enhancing implant placement precision and overall procedure success. Introducing an automated process for report generation through a batch script improves efficiency and precision, streamlining the analysis process. This innovation sets the stage for future technological advancements, improving both the accuracy and reliability of implant placement assessments.

Pose-oriented scene-adaptive matching for abnormal event detection

For intelligent surveillance systems, abnormal event detection automatically analyses surveillance video sequences and detects abnormal objects or unusual human actions at the frame level. Due to the lack of labelled data, most approaches are semi-supervised based on reconstruction or prediction methods. However, these methods may not generalize well to unseen scene contexts. To address this issue, we present a novel self and mutual scene-adaptive matching method for abnormal event detection. In the framework, we propose synergistic pose estimation and object detection, which effectively integrates human pose and object detection information to improve pose estimation accuracy. Then, the poses are resized to reduce the spatial distance between the source and target domains. The improved pose sequences are further fed into a spatio-temporal graph convolutional network to extract the geometric features. Finally, the features are embedded in a clustering layer to classify action types and compute normality scores. The training data is taken from the training part of common video anomaly detection datasets: UCSD PED1 & PED2, CHUK Avenue, and ShanghaiTech Campus. The proposed framework is evaluated on video sequences with unseen scene contexts in the UCSD PED2 and ShanghaiTech Campus datasets. The detection accuracy and efficiency are also evaluated in detail, and the proposed method for abnormal event detection achieves the highest AUC performance, 84.6%, on the ShanghaiTech Campus dataset and relatively high AUC performance, 96.9% and 74.8%, on UCSD PED2 & PED1 datasets. Compared with other state-of-the-art works, the performance analysis and results confirm the robustness and effectiveness of our proposed framework for cross-scene abnormal event detection.

Unveiling digital mirrors: Decoding gendered body poses in instagram imagery

Social media platforms have had a significant impact on people's everyday lives worldwide and imagery on social media has become a vital means of practicing views of the self and communicating them to others. Albeit a significant proportion of people frequently upload self-representations on social media, this source of information on individuals, groups, and societies remains under-examined within the social sciences and neighboring disciplines. In our study, we focus on gender-stereotypical body-posing in self-portraits on social media. While sociology has examined gender stereotypes for decades, research lacks empirical evidence on representations in digital contexts as well as novel forms of stereotyping. We present a scalable and transferable methodology for analyzing body poses in images by combining neural network pose detection with an unsupervised learning approach and applying this methodology to data from the social media platform Instagram. Based on a clustering algorithm applied to gender-annotated imagery, we identify 150 body posing clusters. Our results reveal significant gender differences in 20 percent of clusters, many of which represent gender-stereotypical body poses addressed in sociological literature. Moreover, we can identify new stereotypical poses related to smartphone technology and social media trends. This study represents a novel approach to utilizing large-scale image data for social science research and contributes to a better understanding of the consolidation and reproduction of gender stereotypes in digital realms.

DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions

Fiducial markers are a computer vision tool used for object pose estimation and detection. These markers are highly useful in fields such as industry, medicine and logistics. However, optimal lighting conditions are not always available, and other factors such as blur or sensor noise can affect image quality. Classical computer vision techniques that precisely locate and decode fiducial markers often fail under difficult illumination conditions (e.g. extreme variations of lighting within the same frame). Hence, we propose DeepArUco++, a deep learning-based framework that leverages the robustness of Convolutional Neural Networks to perform marker detection and decoding in challenging lighting conditions. The framework is based on a pipeline using different Neural Network models at each step, namely marker detection, corner refinement and marker decoding. Additionally, we propose a simple method for generating synthetic data for training the different models that compose the proposed pipeline, and we present a second, real-life dataset of ArUco markers in challenging lighting conditions used to evaluate our system. The developed method outperforms other state-of-the-art methods in such tasks and remains competitive even when testing on the datasets used to develop those methods. Code available in GitHub: https://github.com/AVAuco/deeparuco/.

My Child Does More Than Sit for Virtual Read‐Aloud: An Exploratory Human Pose Estimation Study

ABSTRACTVirtual storytime programs supported young children and families when in‐person library services were discontinued in response to the COVID‐19 pandemic in the United States. These virtual storytimes offered via videoconferencing systems allowed for the inclusion of some in‐person storytime elements such as the librarian read‐aloud, and children are the target audience of these picturebook read‐alouds. Children's nonverbal behavior symbolizes a powerful clue as to when they are engaged with the read‐aloud story. What is unclear is how a child nonverbally engages with the storytime readings. This study shows how human pose detection software, OpenPose, was instrumented with custom Python scripts to track and code a preschooler's nonverbal behavior during a virtual storytime read‐aloud. Results demonstrate considerable nonverbal attentiveness and engagement by this child during the read‐aloud.

A high-precision and efficient method for badminton action detection in sports using You Only Look Once with Hourglass Network

Standardized striking movements are essential in badminton for enhancing player techniques and minimizing sports-related injuries. However, accurately detecting these movements against complex backgrounds while balancing precision and speed remains a significant challenge. To address this, we propose a novel model that synergizes You Only Look Once (YOLO) with the Hourglass Network (HGNet), called YOLO-HGNet, to enhance feature learning across multiple levels. By replacing traditional convolutional modules with Depth-Wise Convolution (DWConv), we achieve significant improvements in data processing efficiency. Additionally, our model incorporates a combination of self-attention and convolution mechanisms (ACmix) and FocalModulation to improve object localization and recognition accuracy in complex backgrounds. Our method leverages action vectors and machine learning techniques to accurately detect and classify six key badminton strokes: backhand push, backhand net shot, forehand clear, forehand push, forehand lift, and forehand net shot. Empirical evaluations demonstrate that our approach achieves a mean Average Precision (mAP) of 96.1% for detecting badminton player postures, outperforming existing advanced methods by at least 8.8%. Furthermore, our method achieves an average accuracy of 95.4% in classifying the six badminton strokes. These results underscore the superior capability of YOLO-HGNet for precise and efficient pose detection, recognition, and classification of badminton strokes, contributing significantly to advancements in sports science and athlete training methodologies.

Passive Vision Detection of Torch Pose in Swing Arc Narrow Gap Welding.

To enhance the synchronous detection of the horizontal and vertical positions of the torch in swing arc narrow gap welding, a torch pose detection (TPD) method is proposed. This approach utilizes passive visual sensing to capture images of the arc on the groove sidewall, using advanced image processing methods to extract and fit the arc contour. The coordinates of the arc contour center point and the highest point are determined through the arc contour fitting line. The torch center position is calculated from the average horizontal coordinates of the arc contour centers in adjacent welding images, while the height position is determined from the vertical coordinate of the arc's highest point. Experimental validation in both variable and constant groove welding conditions demonstrated the TPD method's accuracy within 0.32 mm for detecting the torch center position. This method eliminates the need to construct the wire centerline, which was a requirement in previous approaches, thereby reducing the impact of wire straightness on detection accuracy. The proposed TPD method successfully achieves simultaneous detection of the torch center and height positions, laying the foundation for intelligent detection and adaptive control in swing arc narrow gap welding.

A Novel Robotic Pushing and Grasping Method Based on Vision Transformer and Convolution.

Robotic grasping techniques have been widely studied in recent years. However, it is always a challenging problem for robots to grasp in cluttered scenes. In this issue, objects are placed close to each other, and there is no space around for the robot to place the gripper, making it difficult to find a suitable grasping position. To solve this problem, this article proposes to use the combination of pushing and grasping (PG) actions to help grasp pose detection and robot grasping. We propose a pushing-grasping combined grasping network (GN), PG method based on transformer and convolution (PGTC). For the pushing action, we propose a vision transformer (ViT)-based object position prediction network pushing transformer network (PTNet), which can well capture the global and temporal features and can better predict the position of objects after pushing. To perform the grasping detection, we propose a cross dense fusion network (CDFNet), which can make full use of the RGB image and depth image, and fuse and refine them several times. Compared with previous networks, CDFNet is able to detect the optimal grasping position more accurately. Finally, we use the network for both simulation and actual UR3 robot grasping experiments and achieve SOTA performance. Video and dataset are available at https://youtu.be/Q58YE-Cc250.

Selective Grasping for Complex-Shaped Parts Using Topological Skeleton Extraction

To enhance the autonomy and flexibility of robotic systems, a crucial role is played by the capacity to perceive and grasp objects. More in detail, robot manipulators must detect the presence of the objects within their workspace, identify the grasping point, and compute a trajectory for approaching the objects with a pose of the end-effector suitable for performing the task. These can be challenging tasks in the presence of complex geometries, where multiple grasping-point candidates can be detected. In this paper, we present a novel approach for dealing with complex-shaped automotive parts consisting of a deep-learning-based method for topological skeleton extraction and an active grasping pose selection mechanism. In particular, we use a modified version of the well-known Lightweight OpenPose algorithm to estimate the topological skeleton of real-world automotive parts. The estimated skeleton is used to select the best grasping pose for the object at hand. Our approach is designed to be more computationally efficient with respect to other existing grasping pose detection methods. Quantitative experiments conducted with a 7 DoF manipulator on different real-world automotive components demonstrate the effectiveness of the proposed approach with a success rate of 87.04%.

Research on Vehicle Pose Detection Method Based on a Roadside Unit.

Vehicle pose detection plays a vital role in modern automotive technology, which can improve driving safety, enhance vehicle stability and provide important support for the development of autonomous driving technology. The current pose estimation methods have the problems of accumulation errors, large algorithm computing power, and expensive cost, so they cannot be widely used in intelligent connected vehicles. This paper proposes a vehicle pose detection method based on an RSU (Roadside Unit). First, the on-board GPS performs the positioning of the target vehicle and transmits the positioning information to the RSU through the UDP (User Data Protocol). Next, the RSU transmits a forward command to the OBU (On-board Unit) through the UDP. The OBU sends the command to the ECU (Electronic Control Unit) to control the vehicle forward. Then, the RSU detects and tracks the vehicle. The RSU takes pictures of two images before and after the movement and obtains the coordinates of the four angle points and the center point by image processing. The vehicle heading direction is determined by the moving direction of the center point of the front and rear two images. Finally, the RSU captures the vehicle images in real time, performs the process of tracking, rectangular fitting and pose calculation to obtain the pose information and transmits the information to the OBU to complete the whole process of vehicle pose detection and information transmission. Experiments show that the method can realize accurate and efficient detection of vehicle pose, meet the real-time requirements of vehicle pose detection, and can be widely used in intelligent vehicles.

Deep learning pose detection model for sow locomotion

Lameness affects animal mobility, causing pain and discomfort. Lameness in early stages often goes undetected due to a lack of observation, precision, and reliability. Automated and non-invasive systems offer precision and detection ease and may improve animal welfare. This study was conducted to create a repository of images and videos of sows with different locomotion scores. Our goal is to develop a computer vision model for automatically identifying specific points on the sow's body. The automatic identification and ability to track specific body areas, will allow us to conduct kinematic studies with the aim of facilitating the detection of lameness using deep learning. The video database was collected on a pig farm with a scenario built to allow filming of sows in locomotion with different lameness scores. Two stereo cameras were used to record 2D videos images. Thirteen locomotion experts assessed the videos using the Locomotion Score System developed by Zinpro Corporation. From this annotated repository, computational models were trained and tested using the open-source deep learning-based animal pose tracking framework SLEAP (Social LEAP Estimates Animal Poses). The top-performing models were constructed using the LEAP architecture to accurately track 6 (lateral view) and 10 (dorsal view) skeleton keypoints. The architecture achieved average precisions values of 0.90 and 0.72, average distances of 6.83 and 11.37 in pixel, and similarities of 0.94 and 0.86 for the lateral and dorsal views, respectively. These computational models are proposed as a Precision Livestock Farming tool and method for identifying and estimating postures in pigs automatically and objectively. The 2D video image repository with different pig locomotion scores can be used as a tool for teaching and research. Based on our skeleton keypoint classification results, an automatic system could be developed. This could contribute to the objective assessment of locomotion scores in sows, improving their welfare.

Learning accurate and efficient three-finger grasp generation in clutters with an auto-annotated large-scale dataset

With the development of intelligent manufacturing and robotic technologies, the capability of grasping unknown objects in unstructured environments is becoming more prominent for robots with extensive applications. However, current robotic three-finger grasping studies only focus on grasp generation for single objects or scattered scenes, and suffer from high time expenditure to label grasp ground truth, making them incapable of predicting grasp poses for cluttered objects or generating large-scale datasets. To address such limitations, we first introduce a novel three-finger grasp representation with fewer prediction dimensions, which balances the training difficulty and representation accuracy to obtain efficient grasp performance. Based on this representation, we develop an auto-annotation pipeline and contribute a large-scale three-finger grasp dataset (TF-Grasp Dataset). Our dataset contains 222,720 RGB-D images with over 2 billion grasp annotations in cluttered scenes. In addition, we also propose a three-finger grasp pose detection network (TF-GPD), which detects globally while fine-tuning locally to predict high-quality collision-free grasps from a single-view point cloud. In sum, our work addresses the issue of high-quality collision-free three-finger grasp generation in cluttered scenes based on the proposed pipeline. Extensive comparative experiments show that our proposed methodology outperforms previous methods and improves the grasp quality and efficiency in clutters. The superior results in real-world robot grasping experiments not only prove the reliability of our grasp model but also pave the way for practical applications of three-finger grasping. Our dataset and source code will be released.

Detection Method of Manipulator Grasp Pose Based on RGB-D Image

In order to better solve the visual detection problem of manipulator grasping non-cooperative targets, we propose a method of grasp pose detection based on pixel point and feature fusion. By using the improved U2net network as the backbone for feature extraction and feature fusion of the input image, and the grasp prediction layer detects the grasp pose on each pixel. In order to adapt the U2net to grasp pose detection and improve its detection performance, we improve detection speed and control sampling depth by simplifying its network structure, while retaining some shallow features in feature fusion to enhance its feature extraction capability. We introduce depthwise separable convolution in the grasp prediction layer, further fusing the features extracted from the backbone to obtain predictive feature maps with stronger feature expressiveness. FocalLoss is selected as the loss function to solve the problem of unbalanced positive and negative samples in network training. We use the Cornell dataset for training and testing, perform pixel-level labeling on the image, and replace the labels that are not conducive to the actual grasping. This adaptation helps the dataset better suit the network training and testing while meeting the real-world grasping requirements of the manipulator. The evaluation results on image-wise and object-wise are 95.65% and 91.20% respectively, and the detection speed is 0.007 s/frame. We also used the method for actual manipulator grasping experiments. The results show that our method has improved accuracy and speed compared with previous methods, and has strong generalization ability and portability.