Human Skeleton Model Research Articles

Motor delay image recognition based on deep learning and human skeleton model

Children with developmental delays often exhibit gross motor delay during their early childhood. Identifying such delay typically requires standardized assessments, but these assessments are susceptible to subjective interpretation. To enhance objectivity and reliability in these assessments, this study proposes a novel method utilizing an AI model for identifying gross motor delay in children. The proposed method divides the assessment of gross motor skills into two components: static balance and dynamic balance. First, the Yolo target detection model and the Mediapipe human posture detection model are employed for preprocessing to determine skeletal joints and body positioning information. Subsequently, dimensionality reduction techniques are applied to extract key features. A sliding window method is then used to extract action features identifying static balance features of the human body (e.g., standing on one foot, standing on the left foot) and dynamic balance features (e.g., lifting the heels while walking, jumping on a mat). The machine learning classifier and LSTM-CNN are then used to categorize the static and dynamic balance feature data. Experimental results demonstrate that the proposed method achieved accuracy scores of 80 % and 87 % for static and dynamic balancing skills, respectively.

An Image-Based Interactive Training Method of an Upper Limb Rehabilitation Robot

Aimed at the problem of human–machine interaction between patients and robots in the process of using rehabilitation robots for rehabilitation training, this paper proposes a human–machine interactive control method based on an independently developed upper limb rehabilitation robot. In this method, the camera is used as a sensor, the human skeleton model is used to analyse the moving image, and the key points of the human body are extracted. Then, the three-dimensional coordinates of the key points of the human arm are extracted by depth estimation and spatial geometry, and then the real-time motion data are obtained, and the control instructions of the robot are generated from it to realise the real-time interactive control of the robot. This method can not only improve the adaptability of the system to individual patient differences, but also improve the robustness of the system, which is less affected by environmental changes. The experimental results show that this method can realise real-time control of the rehabilitation robot, and that the robot assists the patient to complete the action with high accuracy. The results show that this control method is effective and can be applied to the fields of robot control and robot-assisted rehabilitation training.

LOCALIZATION METHOD OF HUMAN SKELETAL SEQUENCE MOVEMENTS IN ICE SPORTS

A method based on kernel extreme learning machine is proposed for accurate localization and recognition of human bones. In this method, the human skeleton is divided into 16 key nodes based on physiology, and the human skeleton model is formed by connecting lines between the nodes. The sequence images of human actions are taken as input and fed into the input and hidden layers of the extreme learning machine. During the learning and training process, a kernel function is used to construct a kernel matrix to complete weight allocation. The experimental study was conducted on the action sequences of figure skaters, and the results showed that our method achieved an accuracy rate of over 90% in recognizing human bones, with individual sequences achieving an accuracy rate of over 95%, which is superior to convolutional neural network (CNN) and long short-term memory (LSTM) methods.

Research on the Innovation of Physical Education and Physical Education Teaching Based on Big Data Analysis

Abstract In this paper, Openpose is used to process the sports teaching video to get the coordinates data of the human body joint point positions in each frame of the video, and Kalman filter data fusion is used to establish the human skeleton model. According to the results of the division of the five major parts of the human body, after establishing the limb vectors of the human body’s torso, left arm, right arm, left leg, and right leg in three-dimensional spatial coordinates, the distances between the joints of the five human body skeletons based on the DTW posture matching algorithm were used to extract the characteristics of the sports error technical movements. From the demand of sports digital teaching, the design and implementation of sports basic movement teaching evaluation system based on the DTW posture matching algorithm, and the research and analysis of sports teaching under the background of big data. The results show that the IoU values of batting action localization in 6 segments of physical education teaching are 85.6%, 91.6%, 77.7%, 75.1%, 87.4% and 77.7%, respectively, and the average reach 82.5%, i.e., it shows that the research on action localization and recognition based on the DTW posture matching algorithm has a good performance. In the assessment of movement standardization in physical education, the maximum moment of stretching angle corresponds to the moment of hitting the ball, and its value reaches 3.79, i.e., it reflects that the evaluation system of physical education basic movement teaching can accurately determine whether the students’ movements are accurate or not, and make timely feedbacks to carry out the corrections of physical education movements. This study has the potential to enhance students’ interest and performance in sports and contribute to the advancement of digital sports teaching.

Action recognition for sports combined training based on wearable sensor technology and SVM prediction

In the field of sports, coaches have mainly relied on observing the performance of athletes on the spot to formulate suitable training plans for athletes, which has extremely high requirements for the professionalism of coaches. Based on the above requirements, this paper designs a sports action recognition system for sports enthusiasts based on the SVM algorithm optimization model, and for the purpose of verifying the applicability of the system to different sports fields, experiments are carried out on basketball actions and race walking actions. The system uses wearable sensors to capture the motion data of the user, and then analyzes and identifies the user's actions through the SVM algorithm optimization model. By standardizing the user's sports combination training under the system algorithm, the user can improve their training efficiency and reduce the risk of injury. To establish the human body motion model, this paper divides the human skeleton model into five motion branches. The rotation freedom constraints and joint rotation angle range limits are added to the model to ensure the accuracy of the motion analysis. Combining the forward kinematics of the robot and the homogeneous coordinate transformation, the human body joint rotation motion model and the human bone position and posture model are established. In the end, the user can standardize the sports combination training under the system algorithm. In this paper, through the research of wearable sensor technology and sports combined training action recognition, and apply it to practical life, it aims to promote its development and application.

STJA-GCN: A Multi-Branch Spatial–Temporal Joint Attention Graph Convolutional Network for Abnormal Gait Recognition

Early recognition of abnormal gait enables physicians to determine a prompt rehabilitation plan for patients for the most effective treatment and care. The Kinect depth sensor can easily collect skeleton data describing the position of joints in the human body. However, the default human skeleton model of Kinect includes an excessive number of many joints, which limits the accuracy of the gait recognition methods and increases the computational resources required. In this study, we propose an optimized human skeleton model for the Kinect system and streamline the joints using a center-of-mass calculation. We integrate several techniques to propose an end-to-end, spatial–temporal, joint attention graph convolutional network (STJA-GCN) architecture. We conducted experiments with a fivefold cross-validation on two common datasets of information on abnormal gaits to evaluate the performance of the proposed method. The results show that the STJA-GCN achieved 93.17 and 92.08% accuracy on the two datasets, and compared to the original spatial–temporal graph convolutional network (ST-GCN), the recognition accuracy increases by 9.22 and 20.65%, respectively. Overall, the results demonstrate that the STJA-GCN can accurately recognize abnormal gaits and, thus, can support low-cost rehabilitation assessments at community hospitals or in patients’ homes.

Skeleton-Based Fall Detection with Multiple Inertial Sensors Using Spatial-Temporal Graph Convolutional Networks.

The application of wearable devices for fall detection has been the focus of much research over the past few years. One of the most common problems in established fall detection systems is the large number of false positives in the recognition schemes. In this paper, to make full use of the dependence between human joints and improve the accuracy and reliability of fall detection, a fall-recognition method based on the skeleton and spatial-temporal graph convolutional networks (ST-GCN) was proposed, using the human motion data of body joints acquired by inertial measurement units (IMUs). Firstly, the motion data of five inertial sensors were extracted from the UP-Fall dataset and a human skeleton model for fall detection was established through the natural connection relationship of body joints; after that, the ST-GCN-based fall-detection model was established to extract the motion features of human falls and the activities of daily living (ADLs) at the spatial and temporal scales for fall detection; then, the influence of two hyperparameters and window size on the algorithm performance was discussed; finally, the recognition results of ST-GCN were also compared with those of MLP, CNN, RNN, LSTM, TCN, TST, and MiniRocket. The experimental results showed that the ST-GCN fall-detection model outperformed the other seven algorithms in terms of accuracy, precision, recall, and F1-score. This study provides a new method for IMU-based fall detection, which has the reference significance for improving the accuracy and robustness of fall detection.

Research on Simulation System for Human-SRL Collaborative Motion Planning

At present, there is a problem of visual area switching in the existing SRL (supernumerary robotic limb) operation methods. In response to this problem, the authors’ previous work proposed a new SRL operation method called relatively independent operation, and proposed a corresponding software architecture. The purpose of this paper is to solve the theoretical problems and engineering realization problems of the human-SRL skeleton algorithm module in the software architecture. Therefore, modeling, data collection, data processing, and visualization of a human-SRL system are studied in this paper. Firstly, a human-SRL skeleton visualization simulation system is developed. The condition setting, the applications, and the core algorithm of the simulation system are introduced. The core algorithm mainly contains four types of important functions, namely skeleton model building functions, human-SRL data collection functions, human-SRL data processing functions, and skeleton visualization functions. Secondly, the implementation principles of these four functions are described: (1) For the skeleton model building functions, a human-SRL skeleton model is proposed which is an integration of a human skeleton model and an SRL skeleton model. The construction methods of these three skeleton models are described. (2) For the remaining functions, how to collect and process human data, SRL data, human-SRL data, and how to visualize a human-SRL skeleton are described. Finally, the visualization effect of the developed simulation system on human-SRL skeleton movement is verified by experiments, which proves the correctness of the functions in the simulation system.

3D Human Motion Posture Tracking Method Using Multilabel Transfer Learning

To overcome the high position and posture angle tracking error, long tracking loss time and posture tracking update response time, and low fitness problem of traditional human motion posture tracking methods, in this paper, a three-dimensional (3D) human motion posture tracking method using multilabel transfer learning is proposed. According to the human structure composition and degree of freedom constraints, the 3D human joint skeleton model is constructed to generate the 3D human pose image and perform the noise reduction operation. The background difference is used to detect the 3D human moving target. Using multilabel transfer learning, human motion posture features are extracted from joint position and joint angle, and the estimation results of 3D human motion posture are obtained. The tracking error of human motion posture is corrected by three-step search, and the visual 3D human motion posture tracking results are output. The results show that, compared with the traditional human motion posture tracking method, the position and posture angle tracking errors of the proposed method are 2.18 mm and 0.178 deg, respectively. The tracking loss time and posture tracking update response time are shorter, which proves that the proposed method has more advantages in tracking accuracy and higher adaptability.

Analysis of the Style Characteristics of National Dance Based on 3D Reconstruction.

National dance is an important symbol of national spiritual culture, as it embodies each nation's unique history, living habits, ideology, and culture. Many Chinese ethnic groups have developed their own dance forms and styles, each with their own set of charms. Strengthening our understanding of its style and characteristics is critical to our understanding of the national dance. To begin with, this paper proposes a patch-based multiview stereo reconstruction scheme that can accurately reconstruct 3D character models. The movement capture system then collects national dance movement data and uses them to drive a three-dimensional character model that generates dance visual animation. In the human skeleton model, a spatial-temporal HMM (hidden Markov model) is proposed that not only studies the temporal information of human movements but also the spatial information of adjacent connection points. The recognition rate of this algorithm increases to 95.68 percent when the number of setting points is increased to ten and to 98.28 percent when the number of setting points is increased to twenty, according to the research. The experimental results show that the 3D reconstruction model of ethnic dance developed in this paper is capable of meeting the goal of ethnic dance digitalization while being simple to use, being low in cost, and with a good display effect.

Control framework for collaborative robot using imitation learning-based teleoperation from human digital twin to robot digital twin

Despite the deployment of collaborative robots for various industrial processes, their teaching and control remain comparatively difficult tasks compared with general industrial robots. Various imitation learning methods involving the transfer of human poses to a collaborative robot have been proposed. However, most of these methods depend heavily on deep learning-based human recognition algorithms that fail to recognize complicated human poses. To address this issue, we propose an automated/semi-automated vision-based teleoperation framework using human digital twin and a collaborative robot digital twin models. First, a human pose is recognized and reasoned to a human skeleton model using a convolution encoder-decoder architecture. Next, the developed human digital twin model is taught using the skeletons. As human and collaborative robots have different joints and rotation architectures, pose mapping is achieved using the proposed Bezier curve-based smooth approximation. Then, a real collaborative robot is controlled using the developed robot digital twin. Furthermore, the proposed framework works successfully using a human digital twin in the case of recognition failures of human poses. To verify the effectiveness of the proposed framework, transfers of several human poses to a real collaborative robot are tested and analyzed.

Learning cooperative dynamic manipulation skills from human demonstration videos

This article proposes a method for learning and robotic replication of dynamic collaborative tasks from offline videos. The objective is to extend the concept of learning from demonstration (LfD) to dynamic scenarios, benefiting from widely available or easily producible offline videos. To achieve this goal, we decode important dynamic information, such as the Configuration Dependent Stiffness (CDS), which reveals the contribution of arm pose to the arm endpoint stiffness, from a three-dimensional human skeleton model. Next, through encoding of the CDS via Gaussian Mixture Model (GMM) and decoding via Gaussian Mixture Regression (GMR), the robot’s Cartesian impedance profile is estimated and replicated. We demonstrate the proposed method in a collaborative sawing task with leader–follower structure, considering environmental constraints and dynamic uncertainties. The experimental setup includes two Panda robots, which replicate the leader–follower roles and the impedance profiles extracted from a two-persons sawing video.

Three-dimensional Motion Skeleton Reconstruction Algorithm for Gymnastic Dancing Movements

Aiming at the problem of inaccurate matching results in the traditional three-dimensional reconstruction algorithm of gymnastic skeleton, a three-dimensional motion skeleton reconstruction algorithm of gymnastic dance action is proposed. Taking the center of gravity of the human body as the origin, the position of other nodes in the camera coordinate system relative to the center point of the human skeleton model is calculated, and the human skeleton data collection is completed through action division and posture feature calculation. Polynomial density is introduced into the integration of convolution surface, and the human body model of convolution surface is established according to convolution surface. By using the method of binary parameter matching, the accuracy of the matching results is improved, and the three-dimensional skeleton of gymnastic dance movement is reconstructed. The experimental results show that the fitting degree between the proposed method and the actual reconstruction result is 99.8%, and the reconstruction result of this algorithm has high accuracy.

Research on aerobics training posture motion capture based on mathematical similarity matching statistical analysis

Abstract Aiming at the freely editable characteristics of human motion posture, a human skeleton model is extracted, and a human motion posture model library is established. The application of motion capture system in dance training is analysed, and a method based on similarity matching between feature planes is proposed to calculate each model. Parameters of motion data between parts are obtained. After verification, the method has high accuracy and robustness for the analysis of human poses so that dancers can accurately compare the differences with standard dance movements and provide theoretical support for dancers to perform scientific dance training.

Research on Action Recognition Method of Dance Video Image Based on Human-Computer Interaction

Human-computer interaction technology simplifies the complicated procedures, which aims at solving the problems of inadequate description and low recognition rate of dance action, studying the action recognition method of dance video image based on human-computer interaction. This method constructs the recognition process based on human-computer interaction technology, constructs the human skeleton model according to the spatial position of skeleton, motion characteristics of skeleton, and change angles of skeleton, describes the dance posture features by generating skeleton node graph, and extracts the key frames of dance video image by using the clustering algorithm to recognize the dance action. The experimental results show that the recognition rate of this method under different entropy values is not less than 88%. Under the test conditions of complex, dark, bright, and multiuser interference, this method can make the model to describe the dance posture accurately. Furthermore, the average recognition rates are 93.43%, 91.27%, 97.15%, and 89.99%, respectively. It is suitable for action recognition of most dance video images.

What and how well you exercised? An efficient analysis framework for fitness actions

Human action analysis has been an active research area in computer vision, and has many useful applications such as human computer interaction. Most of the state-of-the-art approaches of human action analysis are data-driven and focus on general action recognition. In this paper, we aim to analyze fitness actions with skeleton sequences and propose an efficient and robust fitness action analysis framework. Firstly, fitness actions from 15 subjects are captured and built to a fitness action dataset (Fitness-28). Secondly, skeleton information is extracted and made alignment with a simplified human skeleton model. Thirdly, the aligned skeleton information is transformed to an uniform human center coordinate system with the proposed spatial–temporal skeleton encoding method. Finally, the action classifier and local–global geometrical registration strategy are constructed to analyze the fitness actions. Experimental results demonstrate that our method can effectively assess fitness action, and have a good performance on artificial intelligence fitness system.

Scene image and human skeleton-based dual-stream human action recognition

The dual stream-based human action recognition model offers the advantage of high recognition accuracy, but the algorithm is less robust in case of lighting changes. The human skeleton has a strong ability to express human behavior and actions; however, the scene information is ignored. Drawing on the idea of the dual-stream model, this paper proposes a human skeleton and scene image-based dual-stream model for human action recognition. The motion features are extracted through the spatio-temporal graph convolution of the human skeleton, and a scene recognition model is proposed based on the sparse frame sampling of video and video-level consensus strategy to process the scene video and gather the visual scene information. The proposed model exploits the advantages of skeleton information in motion expression and the superiority of the image in scene presentation. The scene information and spatio-temporal graph convolution-based human skeleton limbs are fused complementarily to achieve human action recognition. Compared to the conventional optical flow-based dual-stream action recognition method, this model is verified by experimenting under unstable light conditions, and the performance of human action recognition is robust and promising.

Research on volleyball action standardization based on 3D dynamic model

We propose a volleyball action standardization method based on 3D dynamic model. Firstly, optical motion capture technology and 3D scanning technology are used to collect the volleyball motion images. Then, the human skeleton model and the appearance model are constructed. Finally, the distance between each joint of the volleyball player's posture and the standard posture is calculated. The results show that the wrong actions made by the players can be accurately recognized by the proposed method. This study also gives some standardized suggestions and may improve players' action learning and training.

Human motion recognition using three-dimensional skeleton model based on RGBD vision system

Human action recognition is one of the most important topics in computer vision. It has a lot of applications in intelligent monitoring, human-computer interaction, virtual reality, motion analysis and video annotation, etc. However, the effects of recognition are usually affected by human occlusion, self-occlusion, ambiguity and so on. In view of the above problems, this article describes an algorithm to recognize human motion using three-dimensional skeleton model based on RGBD vision system. Firstly, the human body three-dimensional skeleton model is established with the Kinect. Next, the normalized relative positions of skeleton joints and joint angles are adopted as features, and the action database containing the color image, the depth image, the skeletal image and the coordinates of the joints is built. Then, the features are trained and classified by multi-class support vector machine classifier. Finally, human motion recognitions are carried out. Experimental results indicate that the algorithm can recognize 12 kinds of actions behaved by different humans effectively.

Development of a novel real-time simulation of human skeleton/muscles

The objective of this research work was to develop a model of human skeleton with the capability of real-time simulation of the physical movements of a person in front of the motion capture hardware (Kinect) in order to analyze the motion data and measure the changes of joint torques. Mevea simulation software has been utilized for this purpose, which is a novel application of this software in the field of biomechanics. The model of the human skeleton was created in Mevea using the graphics built in 3ds Max. Simulink external interface for Mevea was established. Simulink acts as a connection between the Mevea software and Kinect for controlling the model. The developed model has been tested through three case studies involving the elbow joint, thoracic joint, and full body. Changes in torque and angular position of joints based on the input of joints are presented as graphs. The developed real-time model of the human skeleton in Mevea can execute the real-time simulation of a person's movements in front of a motion capture camera and provide the changes of torques, which are dependent on the angular positions of the body joints. This work provides the possibility to use the developed real-time model for physiotherapeutic rehabilitation to identify problematic muscles based on produced torque of the joints in order to specify the therapeutic options. The future research direction would be creating a reference databank by measuring healthy individuals' muscle forces for comparison purposes.