Human Pose Recognition Research Articles

A Multi-Scale and Multi-Stage Human Pose Recognition Method Based on Convolutional Neural Networks for Non-Wearable Ergonomic Evaluation

In the context of industrial robot maintenance and assembly, workers often suffer from work-related musculoskeletal disorders (WRMSDs). This paper proposes a multi-scale, multi-stage pose recognition method (MMARM-CNN) based on convolutional neural networks to provide ergonomic intervention. The method leverages computer vision technology to enable non-contact data acquisition, reducing the interference of physiological and psychological factors on assessment results. Built upon the baseline yolov8-pose framework, the method addresses complex maintenance environments, which are prone to occlusion, by introducing the Lightweight Shared Convolutional Detection Head-pose (LSCD-pose) module, Multi-Scale Channel Attention (MSCA) mechanism, and Efficient Multi-Scale Patch Convolution (EMSPC) module, enhancing the model’s feature extraction capabilities. The MMARM-CNN model was validated using the MS COCO 2017 dataset and robot assembly data collected under laboratory conditions. The experimental results show that the MMARM-CNN achieved an accuracy improvement, reaching 0.875 in the mAP@0.5 evaluation. Overall, this method demonstrates significant potential in advancing the automation and intelligence of ergonomic interventions.

The Digital Transformation of Functional Movement Screen through Human Pose Recognition based on Artificial Intelligence

The Digital Transformation of Functional Movement Screen through Human Pose Recognition based on Artificial Intelligence

Research on entertainment robots based on artificial intelligence interaction for human posture recognition and sports activity monitoring

With the application of artificial intelligence technology in sports activity monitoring, new technologies such as interactive robots and the Internet of Things are being more widely used, which can provide entertainment and interactive guidance for sports training. This article analyzes the research on human pose recognition and sports activity monitoring of entertainment robots based on artificial intelligence interaction. This paper designs and develops an entertainment robot system that can recognize human posture and monitor physical activities. Users can interact with the robot and get real-time feedback about their posture and activities, thereby improving motor skills and physical fitness. After in-depth analysis of the theory of dynamic Bayesian network, we used EM algorithm to build a dynamic Bayesian network model, carried out detailed research on the random variables represented by the nodes of the dynamic model, and used the Internet of Things human posture recognition technology to extract the features of human behavior according to the characteristics of human behavior. The results show that the attitude recognition accuracy can be improved by adding the dynamic Bayesian network based on the random forest algorithm. The entertainment robot can accurately identify human posture and monitor sports activities, and has good real-time and stability. Through interaction with the robot, users can get personalized suggestions for individual postures and activities, improving the training effect and enjoyment of physical activities. Therefore, through the in-depth research and analysis of the dynamic Bayesian network and the human posture recognition of the Internet of Things, it is found that the sports activity monitoring system proposed in this paper can comprehensively monitor students’ sports activities, so as to achieve the goal of comprehensively improving students’ physical fitness.

Research on Human Posture Estimation Algorithm Based on YOLO-Pose.

In response to the numerous challenges faced by traditional human pose recognition methods in practical applications, such as dense targets, severe edge occlusion, limited application scenarios, complex backgrounds, and poor recognition accuracy when targets are occluded, this paper proposes a YOLO-Pose algorithm for human pose estimation. The specific improvements are divided into four parts. Firstly, in the Backbone section of the YOLO-Pose model, lightweight GhostNet modules are introduced to reduce the model's parameter count and computational requirements, making it suitable for deployment on unmanned aerial vehicles (UAVs). Secondly, the ACmix attention mechanism is integrated into the Neck section to improve detection speed during object judgment and localization. Furthermore, in the Head section, key points are optimized using coordinate attention mechanisms, significantly enhancing key point localization accuracy. Lastly, the paper improves the loss function and confidence function to enhance the model's robustness. Experimental results demonstrate that the improved model achieves a 95.58% improvement in mAP50 and a 69.54% improvement in mAP50-95 compared to the original model, with a reduction of 14.6 M parameters. The model achieves a detection speed of 19.9 ms per image, optimized by 30% and 39.5% compared to the original model. Comparisons with other algorithms such as Faster R-CNN, SSD, YOLOv4, and YOLOv7 demonstrate varying degrees of performance improvement.

Deep multimodal spatio-temporal Harris Hawk Optimized Pose Recognition framework for self-learning fitness exercises

Human pose recognition from videotapes has become an emerging research topic for tracking human movements. The objective of this work is to develop a deep multimodal Spatio-Temporal Harris Hawk Optimized Pose Recognition (STHHO-PR) framework for self-learning fitness exercises. The presented STHHO-PR framework uses audio modality and visual modality to classify the different poses. In audio modality, the VGG-16 network paradigm is used to extract the audio traits for fitness pose recognition. In visual modality, Harris Hawks Optimization (HHO) along with the Minimum Cross Entropy (MCE) method is employed to find out the optimum threshold values for body parts segmentation. These segmented body parts highlight the human joint points that are connected through the skeletonization process to extract the skeletal information. The extracted spatio-temporal features from audio modality and visual modality are optimally fused and used in the classification process. Weighted Majority Voting Ensemble (WMVE) classifier is adopted to build the classification model. This work is experimented with yoga videos acquired from publicly available datasets. The results show that the presented STHHO-PR framework outperforms other state-of-art procedures in terms of prediction accuracy.

Intelligent robotic arm for human pose recognition based on teleoperation system

With the rapid development of skeleton recognition, machine learning and other technologies, we will find that there are great drawbacks in the control of manipulators. Robot teleoperation refers to the inclusion of human operation in the control loop of robot control. When robots deal with complex perception and a large number of tasks, teleoperation is far superior to intelligent programming when making decisions quickly and dealing with extreme situations. The goal of this paper is to build a robotic arm teleoperation system for human motion capture, so as to solve the problems that the control accuracy of the end of the robotic arm is not high and the motion of the robotic arm is greatly affected by the difference between the human arm in the current related research, the master-slave human motion mapping algorithm is designed and extended with machine learning algorithms. We use inertial motion capture to realize teleoperation, so as to avoid the use of the terminal position and orientation control method of the hand controller to form the control command of the remote robot after tedious calculation, and it is convenient for the operator to complete the attitude tracking task in real time. The obtained attitude information has a larger range, higher sensitivity and better dynamic performance.

Fall Detection System Based on Point Cloud Enhancement Model for 24 GHz FMCW Radar.

Automatic fall detection plays a significant role in monitoring the health of senior citizens. In particular, millimeter-wave radar sensors are relevant for human pose recognition in an indoor environment due to their advantages of privacy protection, low hardware cost, and wide range of working conditions. However, low-quality point clouds from 4D radar diminish the reliability of fall detection. To improve the detection accuracy, conventional methods utilize more costly hardware. In this study, we propose a model that can provide high-quality three-dimensional point cloud images of the human body at a low cost. To improve the accuracy and effectiveness of fall detection, a system that extracts distribution features through small radar antenna arrays is developed. The proposed system achieved 99.1% and 98.9% accuracy on test datasets pertaining to new subjects and new environments, respectively.

Surface and underwater human pose recognition based on temporal 3D point cloud deep learning

Airborne surface and underwater human pose recognition are crucial for various safety and surveillance applications, including the detection of individuals in distress or drowning situations. However, airborne optical cameras struggle to achieve simultaneous imaging of the surface and underwater because of limitations imposed by visible-light wavelengths. To address this problem, this study proposes the use of light detection and ranging (LiDAR) to simultaneously detect humans on the surface and underwater, whereby human poses are recognized using a neural network designed for irregular data. First, a temporal point-cloud dataset was constructed for surface and underwater human pose recognition to enhance the recognition of comparable movements. Subsequently, radius outlier removal (ROR) and statistical outlier removal (SOR) were employed to alleviate the impact of noise and outliers in the constructed dataset. Finally, different combinations of secondary sampling methods and sample sizes were tested to improve recognition accuracy using PointNet++. The experimental results show that the highest recognition accuracy reached 97.5012%, demonstrating the effectiveness of the proposed human pose detection and recognition method.

Active disturbance rejection control for unmanned tracked vehicles in leader–follower scenarios: Discrete-time implementation and field test validation

This paper presents a systematic design of an active disturbance rejection control (ADRC) system for unmanned tracked vehicles (UTVs) in leader-follow formation. Two ADRC controllers are designed for the lateral and the longitudinal channels of the UTV based on control errors in the cross-track and the along-track directions. Through simulations, the proposed ADRC approach is first shown to outperform the conventional PI/PID controllers in scenarios involving sudden changes in the leader motion dynamics, slippage disturbances, and measurement noise. Then, a comprehensive experimental validation of the proposed leader–follower control is performed using a laboratory UTV equipped with a camera and laser sensors (to enable the calculation of error signals). In order to provide more effective interaction between the human (leader) and the UTV (follower) during the leader–follower task, a camera-based subsystem for human pose recognition is developed and deployed. Finally, the experimental results obtained outdoors demonstrate that the proposed ADRC-based leader–follower UTV control system achieves high tracking capabilities, robustness against slippage disturbances, and adaptability to changing environmental conditions.

Recognition Method with Deep Contrastive Learning and Improved Transformer for 3D Human Motion Pose

Three-dimensional (3D) human pose recognition techniques based on spatial data have gained attention. However, existing models and algorithms fail to achieve desired precision. We propose a 3D human motion pose recognition method using deep contrastive learning and an improved Transformer. The improved Transformer removes noise between human motion RGB and depth images, addressing orientation correlation in 3D models. Two-dimensional (2D) pose features are extracted from de-noised RGB images using a kernel generation module in a graph convolutional network (GCN). Depth features are extracted from de-noised depth images. The 2D pose features and depth features are fused using a regression module in the GCN to obtain 3D pose recognition results. The results demonstrate that the proposed method captures RGB and depth images, achieving high recognition accuracy and fast speed. The proposed method demonstrates good accuracy in 3D human motion pose recognition.

An optimization high-resolution network for human pose recognition based on attention mechanism

An optimization high-resolution network for human pose recognition based on attention mechanism

A Hybrid Human Activity Recognition Method Using an MLP Neural Network and Euler Angle Extraction Based on IMU Sensors

Inertial measurement unit (IMU) technology has gained popularity in human activity recognition (HAR) due to its ability to identify human activity by measuring acceleration, angular velocity, and magnetic flux in key body areas like the wrist and knee. It has propelled the extensive application of HAR across various domains. In the healthcare sector, HAR finds utility in monitoring and assessing movements during rehabilitation processes, while in the sports science field, it contributes to enhancing training outcomes and preventing exercise-related injuries. However, traditional sensor fusion algorithms often require intricate mathematical and statistical processing, resulting in higher algorithmic complexity. Additionally, in dynamic environments, sensor states may undergo changes, posing challenges for real-time adjustments within conventional fusion algorithms to cater to the requirements of prolonged observations. To address these limitations, we propose a novel hybrid human pose recognition method based on IMU sensors. The proposed method initially calculates Euler angles and subsequently refines them using magnetometer and gyroscope data to obtain the accurate attitude angle. Furthermore, the application of FFT (Fast Fourier Transform) feature extraction facilitates the transition of the signal from its time-based representation to its frequency-based representation, enhancing the practical significance of the data. To optimize feature fusion and information exchange, a group attention module is introduced, leveraging the capabilities of a Multi-Layer Perceptron which is called the Feature Fusion Enrichment Multi-Layer Perceptron (GAM-MLP) to effectively combine features and generate precise classification results. Experimental results demonstrated the superior performance of the proposed method, achieving an impressive accuracy rate of 96.13% across 19 different human pose recognition tasks. The proposed hybrid human pose recognition method is capable of meeting the demands of real-world motion monitoring and health assessment.

Detection of Anomalous Behavior of Manufacturing Workers Using Deep Learning-Based Recognition of Human–Object Interaction

The increasing demand for industrial products has expanded production quantities, leading to negative effects on product quality, worker productivity, and safety during working hours. Therefore, monitoring the conditions in manufacturing environments, particularly human workers, is crucial. Accordingly, this study presents a model that detects workers’ anomalous behavior in manufacturing environments. The objective is to determine worker movements, postures, and interactions with surrounding objects based on human–object interactions using a Mask R-CNN, MediaPipe Holistic, a long short-term memory (LSTM), and worker behavior description algorithm. The process begins by recognizing the objects within video frames using a Mask R-CNN. Afterward, worker poses are recognized and classified based on object positions using a deep learning-based approach. Next, we identified the patterns or characteristics that signified normal or anomalous behavior. In this case, anomalous behavior consists of anomalies correlated with human pose recognition (emergencies: worker falls, slips, or becomes ill) and human pose recognition with object positions (tool breakage and machine failure). The findings suggest that the model successfully distinguished anomalous behavior and attained the highest pose recognition accuracy (approximately 96%) for standing, touching, and holding, and the lowest accuracy (approximately 88%) for sitting. In addition, the model achieved an object detection accuracy of approximately 97%.

Nested Densely Atrous Spatial Pyramid Pooling and Deep Dense Short Connection for Skeleton Detection

The skeleton shows the local symmetry and the shape/topology of the object, and it is utilized for human pose recognition, road detection, text detection, and the representation of industrial parts. However, the size of the skeleton is variable, which makes high-level feature representation difficult. Existing methods only attempt to integrate multilevel features but ignore the extraction of high-level features and multiscales of contextual information that are helpful for the skeleton detection task. Thus, the contributions of this article include two aspects. The first contribution is to propose a nested densely atrous spatial pyramid pooling method that connects the atrous convolutions with different dilation rates in a nested cascade mode, which can provide the multiscale denser contextual information, a larger receptive field, and more local features. The second one is to propose a deep dense short connection (DDSC) that explores the role of features at different levels in the task of skeleton detection. DDSC adopts concatenation to fuse high-level semantic information with shape information. The proposed method is evaluated on four common datasets, and the experimental results show the effectiveness of the proposed method.

Radio-Frequency-Identification-Based 3D Human Pose Estimation Using Knowledge-Level Technique

Human pose recognition is a new field of study that promises to have widespread practical applications. While there have been efforts to improve human position estimation with radio frequency identification (RFID), no major research has addressed the problem of predicting full-body poses. Therefore, a system that can determine the human pose by analyzing the entire human body, from the head to the toes, is required. This paper presents a 3D human pose recognition framework based on ANN for learning error estimation. A workable laboratory-based multisensory testbed has been developed to verify the concept and validation of results. A case study was discussed to determine the conditions under which an acceptable estimation rate can be achieved in pose analysis. Using the Butterworth filtering technique, environmental factors are de-noised to reduce the system’s computational cost. The acquired signal is then segmented using an adaptive moving average technique to determine the beginning and ending points of an activity, and significant features are extracted to estimate the activity of each human pose. Experiments demonstrate that RFID transceiver-based solutions can be used effectively to estimate a person’s pose in real time using the proposed method.

Task Offloading in Multi-Access Edge Computing Enabled UAV-Aided Emergency Response Operations

In emergency response operations, using uncrewed aerial vehicles (UAVs) has recently become a promising solution due to their flexibility and easy deployment. However, tasks performed by the UAVs, e.g., object detection and human pose recognition, usually require a high computation capacity and energy supply. Furthermore, offloading tasks to the edge server-equipped base stations may not always be possible because of a lack of infrastructure or distance. Therefore, UAV-aided edge servers can be deployed near UAV scouts to provide computing services. However, a UAV can not perform all types of tasks since it has limitations on memory, available software, central processing unit (CPU), and graphics processing unit (GPU) capacity. Therefore, this study focuses on task offloading (TO), power, and computation resource allocation (PRA) problems in a multi-layer MEC-enabled UAV network while taking into account CPU and GPU requirements of tasks, the capacity of the devices (i.e., computational resources, power, and energy), and limitations on the type of tasks a UAVs can perform. The problem is formulated as a non-convex mixed-integer nonlinear problem to minimize the weighted sum of the maximum energy consumption ratio in the network and total task execution latency ratio, and then decomposed and converted into an integer and a convex problem. A messy genetic algorithm (mGA)-based TO and PRA strategy (mGA-TPR) is proposed to solve the problem, where two PRA strategies are based on the Karush–Kuhn–Tucker conditions used to solve the PRA problem. Simulation results verify that the proposed scheme can outperform the baseline methods.

Study on Volleyball-Movement Pose Recognition Based on Joint Point Sequence.

With the high-speed operation of society and the increasing development of modern science, people's quality of life continues to improve. Contemporary people are increasingly concerned about their quality of life, pay attention to body management, and strengthen physical exercise. Volleyball is a sport that is loved by many people. Studying volleyball postures and recognizing and detecting them can provide theoretical guidance and suggestions for people. Besides, when it is applied to competitions, it can also help the judges to make fair and reasonable decisions. At present, pose recognition in ball sports is challenging in action complexity and research data. Meanwhile, the research also has an important application value. Therefore, this article studies human volleyball pose recognition by combining the analysis and summary of the existing human pose recognition studies based on joint point sequences and long short-term memory (LSTM). This article proposes a data preprocessing method based on the angle and relative distance feature enhancement and a ball-motion pose recognition model based on LSTM-Attention. The experimental results show that the data preprocessing method proposed here can further improve the accuracy of gesture recognition. For example, the joint point coordinate information of the coordinate system transformation significantly improves the recognition accuracy of the five ball-motion poses by at least 0.01. In addition, it is concluded that the LSTM-attention recognition model is not only scientific in structure design but also has considerable competitiveness in gesture recognition performance.

Skeleton-Based Human Pose Recognition Using Channel State Information: A Survey.

With the increasing demand for human-computer interaction and health monitoring, human behavior recognition with device-free patterns has attracted extensive attention. The fluctuations of the Wi-Fi signal caused by human actions in a Wi-Fi coverage area can be used to precisely identify the human skeleton and pose, which effectively overcomes the problems of the traditional solution. Although many promising results have been achieved, no survey summarizes the research progress. This paper aims to comprehensively investigate and analyze the latest applications of human behavior recognition based on channel state information (CSI) and the human skeleton. First, we review the human profile perception and skeleton recognition progress based on wireless perception technologies. Second, we summarize the general framework of precise pose recognition, including signal preprocessing methods, neural network models, and performance results. Then, we classify skeleton model generation methods into three categories and emphasize the crucial difference among these typical applications. Furthermore, we discuss two aspects, such as experimental scenarios and recognition targets. Finally, we conclude the paper by summarizing the issues in typical systems and the main research directions for the future.

Image and Speech Recognition Technology in the Development of an Elderly Care Robot: Practical Issues Review and Improvement Strategies.

As the world's population is aging and there is a shortage of sufficient caring manpower, the development of intelligent care robots is a feasible solution. At present, plenty of care robots have been developed, but humanized care robots that can suitably respond to the individual behaviors of elderly people, such as pose, expression, gaze, and speech are generally lacking. To achieve the interaction, the main objectives of this study are: (1) conducting a literature review and analyzing the status quo on the following four core tasks of image and speech recognition technology: human pose recognition, human facial expression recognition, eye gazing recognition, and Chinese speech recognition; (2) proposing improvement strategies for these tasks based on the results of the literature review. The results of the study on these improvement strategies will provide the basis for using human facial expression robots in elderly care.

Research on Animation Character Based on the Semantic Segmentation Model

A semantic ontology-driven hierarchical consistency segmentation algorithm was proposed to solve the segmentation inconsistency of animation character models because of the changing of poses. The mapping between semantic labels and local geometric features was extracted to form a segmentation ontology. In the process of segmentation, support vector machine (SVM) and local geometric features were used to identify the semantic labels, and segmentation was carried out according to the semantic label driving hierarchy to ensure the consistency of the segmentation levels of animation character models. Poisson equation was used to define contour lines for the equal perimeters of segmentation boundary because of the changing of poses. This optimization method made the segmentation boundary smooth and consistent under the changing of poses. In the experiment part, all kinds of animation character models under different poses were verified and analyzed, and the consistent hierarchical segmentation effect was obtained. Compared with the existing methods, the proposed segmentation ontology can solve the problem of adaptive selection of optimization parameters of different classes of models, and improve the segmentation quality. With the continuous development of deep learning, the use of image segmentation for animation, and human pose recognition will become more and more important.