Skeleton Motion Research Articles

Recognition of Hitting Action in Cyclic Anaerobic Volleyball by Acute Cooling Based on Improved Spatiotemporal Graph Convolutional Network

Action recognition is the basis of intelligent sports training and has a unique role in improving athletes’ sports training ability. Traditional motion recognition cannot accurately identify changes in human skeletal motion, and the generalization ability is weak, which cannot meet the needs of modern sports. This paper is aimed at studying a new method of human motion recognition and better realizing the intelligent development of the Institute of Physical Education. This paper proposes an improved recognition mode of spatiotemporal graph convolutional network, builds a human skeleton feature recognition model, learns the changes of human skeleton motion, and masters the laws of human skeleton motion changes. This paper comprehensively compares the differences of action recognition of several different algorithms and proves the superiority of this method. Afterwards, a comprehensive comparison of 2D and 3D information data and a comparison of whole body and local action recognition methods were conducted to investigate the experience of athletes and trainers. The experimental results of this paper show that the spatiotemporal graph convolutional network can better realize the recognition of motion features, improve the sports training ability of athletes by 20%, and promote the development of sports.

Analysis of Students' Sports Exercise Behavior and Health Education Strategy Using Visual Perception-Motion Recognition Algorithm.

This study aims to explore the future development path of the college health education and health education's impact on students' sports exercise. Specifically, artificial intelligence (AI) algorithm is combined with intelligent robotics technology to acquire and analyze students' sports exercise behaviors. As a result, a new development model is formulated for college health education. First, it explores students' sports exercise and health education situation in Chinese higher institutions and uncovers the underlying problems. Then it puts forward the corresponding modification suggestions. Second, the AI algorithm and the Kinect sensor-mounted intelligent robot capture the human skeleton features to obtain smooth skeleton joint points data. At the same time, a visual perception human motion recognition (HMR) algorithm is established based on the Hidden Markov Model (HMM). Afterward, the proposed HMM-based HMR algorithm is used to recognize students' sports exercise motions by analyzing human motion skeleton images. The experimental outcomes suggest that the maximum reconstruction error of the HMR algorithm is 10 mm, and the compression ratio is between 5 and 10; the HMR rate is more than 96%. Compared with similar algorithms, the proposed visual perception HMR algorithm depends less on the number of training samples. It can achieve a high recognition rate given only a relatively few samples. Therefore, the proposed (AI + intelligent robot)-enabled HMM-based HMR algorithm can effectively identify the behavior characteristics of students in sports exercise. This study can provide a reference for exploring college students' health education development path.

An iterative solution for improving the generalization ability of unsupervised skeleton motion retargeting

In this paper, a deep learning-based iterative solving method for unsupervised motion retargeting is proposed. Unlike existing methods, we do not utilize generative-adversarial-network-based (GAN-based) models as a solution for unsupervised motion retargeting, but utilize an iterative solution to yield retargeted motions based on a trained motion autoencoder. The proposed method includes a training phase and an execution phase. In the training phase, an autoencoder named the iterative motion retargeting network (ItMRnet) learns a continuous latent space of natural motion data. At the execution phase, the autoencoder-based hidden units are iteratively adjusted to yield retargeted motion sequences by minimizing the loss functions defined in motion space. Existing GAN-based methods need to train discriminators to distinguish whether the output of the generator is real or fake motion data. Obviously, the discriminator would have a poor discrimination ability for characters and motions that are not included in the training dataset. The proposed method does not use adversarial training which hurt generalization ability, but yields motion from a continuous latent space that represents a motion manifold, and can generate motions for those characters not included in the training dataset. That is, our method generalizes well to unseen motions and characters. The experimental results demonstrate that our method clearly outperforms state-of-the-art methods by reducing the motion retargeting error and motion deformation phenomenon, especially when the source or target character is not seen in the training dataset.

Transient wave-induced pore-water-pressure and soil responses in a shallow unsaturated poroelastic seabed

An analytical solution is developed for studying transient water wave-induced responses inside an unsaturated poroelastic seabed of finite thickness. The soil skeleton and the pore fluid are compressible and the constitutive relationship of the soil skeleton is described by Hooke's law. Assuming that the horizontal length scale of wave motion is much larger than the seabed thickness, the leading-order analytical solutions for the seabed responses, including pore fluid pressure and soil skeleton motion, are obtained. The present solutions are suitable for general transient wave loading and for the shear modulus of the soil skeleton being of the same order of magnitude as the effective bulk modulus of elasticity of the pore fluid. The present theory is first validated by checking the solutions with the experimental data for the pore pressure induced by periodic-wave loading. The present analytical solutions are then used to investigate the seabed responses under transient waves, including linear periodic wave, a solitary wave and a bore. The effects of the wave-induced effective stresses on the bed failure potential are further analysed. The results show that the shear failure potential and its duration are highly dependent on the soil properties, such as saturation degree, shear modulus and permeability. Sensitivity analyses are presented.

Three‐dimensional human activity recognition by forming a movement polygon using posture skeletal data from depth sensor

Human activity recognition in real time is a challenging task. Recently, a plethora of studies has been proposed using deep learning architectures. The implementation of these architectures requires the high computing power of the machine and a massive database. However, handcrafted features-based machine learning models need less computing power and very accurate where features are effectively extracted. In this study, we propose a handcrafted model based on three-dimensional sequential skeleton data. The human body skeleton movement over a frame is computed through joint positions in a frame. The joints of these skeletal frames are projected into two-dimensional space, forming a “movement polygon.” These polygons are further transformed into a one-dimensional space by computing amplitudes at different angles from the centroid of polygons. The feature vector is formed by the sampling of these amplitudes at different angles. The performance of the algorithm is evaluated using a support vector machine on four public datasets: MSR Action3D, Berkeley MHAD, TST Fall Detection, and NTU-RGB+D, and the highest accuracies achieved on these datasets are 94.13%, 93.34%, 95.7%, and 86.8%, respectively. These accuracies are compared with similar state-of-the-art and show superior performance.

Three-dimensional Motion Skeleton Reconstruction Algorithm for Gymnastic Dancing Movements

Aiming at the problem of inaccurate matching results in the traditional three-dimensional reconstruction algorithm of gymnastic skeleton, a three-dimensional motion skeleton reconstruction algorithm of gymnastic dance action is proposed. Taking the center of gravity of the human body as the origin, the position of other nodes in the camera coordinate system relative to the center point of the human skeleton model is calculated, and the human skeleton data collection is completed through action division and posture feature calculation. Polynomial density is introduced into the integration of convolution surface, and the human body model of convolution surface is established according to convolution surface. By using the method of binary parameter matching, the accuracy of the matching results is improved, and the three-dimensional skeleton of gymnastic dance movement is reconstructed. The experimental results show that the fitting degree between the proposed method and the actual reconstruction result is 99.8%, and the reconstruction result of this algorithm has high accuracy.

A Two-stream Convolutional Network for Musculoskeletal and Neurological Disorders Prediction

Musculoskeletal and neurological disorders are the most common causes of walking problems among older people, and they often lead to diminished quality of life. Analyzing walking motion data manually requires trained professionals and the evaluations may not always be objective. To facilitate early diagnosis, recent deep learning-based methods have shown promising results for automated analysis, which can discover patterns that have not been found in traditional machine learning methods. We observe that existing work mostly applies deep learning on individual joint features such as the time series of joint positions. Due to the challenge of discovering inter-joint features such as the distance between feet (i.e. the stride width) from generally smaller-scale medical datasets, these methods usually perform sub-optimally. As a result, we propose a solution that explicitly takes both individual joint features and inter-joint features as input, relieving the system from the need of discovering more complicated features from small data. Due to the distinctive nature of the two types of features, we introduce a two-stream framework, with one stream learning from the time series of joint position and the other from the time series of relative joint displacement. We further develop a mid-layer fusion module to combine the discovered patterns in these two streams for diagnosis, which results in a complementary representation of the data for better prediction performance. We validate our system with a benchmark dataset of 3D skeleton motion that involves 45 patients with musculoskeletal and neurological disorders, and achieve a prediction accuracy of 95.56%, outperforming state-of-the-art methods.

An Innovative Project of a Bionic Robot for Social Applications in Felinotherapy

The paper describes an innovative design of a bionic robot for applications in felinotherapy supporting hospital and home psychotherapeutic treatment of bedridden children and adults. The project was engineered by biomimicrating a biological cat, reaching its robotic model. Particular attention in this process was devoted to capturing the essence of feline motorics behavior and the possibility of mapping them in a mechatronic model. The geometry, kinematics and kinetics of this model were analyzed, creating assumptions for its practical implementation in the real mechanism of cat skeleton movement. The used software used the topology of elements in Autodesk Fusion 360 Simulation workspace by performing the critical elements of the mechatronic model in print using SLS technology. The work was also supported by a graphical simulation in the PyBullet environment.

Fusion of Skeleton and Inertial Data for Human Action Recognition Based on Skeleton Motion Maps and Dilated Convolution

Human activity recognition (HAR) has become a hot research topic, due to its wide application prospect. Fusion-based methods can be used to complement single sensing modality methods. This paper presents the simultaneous utilization of skeleton data and inertial signals—which are captured at the same time using a Kinect depth camera and ten wearable inertial sensors —within a fusion framework, in order to achieve more robust human action recognition, compared to situations where each sensing modality is used individually. Skeleton data captured by the Kinect depth camera are transformed into a weighted front-view skeleton motion map (WF-SMM), a weighted multi-view skeleton motion map (WM-SMM), and a 3D weighted skeleton motion map (3DW-SMM), which are then fed as inputs into a convolutional neural network. Meanwhile, the inertial data are transformed into 2D inertial images, then fed into a 2D dilated convolutional neural network. Two types of fusion are considered: decision-level fusion and feature-level fusion. Experiments were conducted using the publicly available Changzhou University Multimodal Human Action Data set (CZU-MHAD), in which simultaneous skeleton sequence and inertial signals were captured for a total of 22 actions. The results obtained indicate that both the decision- and feature-level fusion approaches generate higher recognition accuracies, compared to the approaches where each sensing modality is used individually. The highest accuracy (of 98.90%) was obtained with the decision-level fusion approach using 3DW-SMM. In addition, some experiments are conducted on the continuous action streams generated based on the CZU-MHAD with different score threshold. And the highest f1 score 82.05 % is obtained with the threshold of 0.4.

Hydro-mechanical coupled B-spline material point method for large deformation simulation of saturated soils

Numerical simulation for large deformation problems of saturated soil poses an ongoing challenge since it requires the capabilities of modeling large deformations, interactions between solid skeleton and pore water, as well as fluidization behaviors. A hydro-mechanical coupled B-spline material point method is developed to simulate large deformations of saturated soils. Based on Biot's mixture theory, a single set of material particles is adopted to represent saturated soils where Darcy's law is employed to govern the motion of pore water and the momentum conservation of the mixture is used to govern the motion of solid skeleton. The convergence and effectiveness of the proposed method are examined by one-dimensional consolidation tests and quasi-static footing penetration into two-dimensional elastic saturated soils. The capability of simulating practical engineering for the proposed method is demonstrated by reproducing the dynamic process of three-dimensional saturated soil slopes with a strain-softening Drucker-Prager constitutive model. Results show that the coupled BSMPM well reveals the overall dynamic process of saturated soil slope involving evolution of shear bands, formation of graben and horst, and fluidization of progressive failures with improved computational accuracy and enhanced convergence, which demonstrates the proposed method is a promising approach for large deformation problems of saturated soils.

Real-time deep dynamic characters

We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery. In contrast to previous work, our controllable 3D character displays dynamics, e.g., the swing of the skirt, dependent on skeletal body motion in an efficient data-driven way, without requiring complex physics simulation. Our character model also features a learned dynamic texture model that accounts for photo-realistic motion-dependent appearance details, as well as view-dependent lighting effects. During training, we do not need to resort to difficult dynamic 3D capture of the human; instead we can train our model entirely from multi-view video in a weakly supervised manner. To this end, we propose a parametric and differentiable character representation which allows us to model coarse and fine dynamic deformations, e.g., garment wrinkles, as explicit spacetime coherent mesh geometry that is augmented with high-quality dynamic textures dependent on motion and view point. As input to the model, only an arbitrary 3D skeleton motion is required, making it directly compatible with the established 3D animation pipeline. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing, including dynamics, and a neural generative dynamic texture model creates corresponding dynamic texture maps. We show that by merely providing new skeletal motions, our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches, and even in real-time.

Real-time deep dynamic characters

We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery. In contrast to previous work, our controllable 3D character displays dynamics, e.g., the swing of the skirt, dependent on skeletal body motion in an efficient data-driven way, without requiring complex physics simulation. Our character model also features a learned dynamic texture model that accounts for photo-realistic motion-dependent appearance details, as well as view-dependent lighting effects. During training, we do not need to resort to difficult dynamic 3D capture of the human; instead we can train our model entirely from multi-view video in a weakly supervised manner. To this end, we propose a parametric and differentiable character representation which allows us to model coarse and fine dynamic deformations, e.g., garment wrinkles, as explicit spacetime coherent mesh geometry that is augmented with high-quality dynamic textures dependent on motion and view point. As input to the model, only an arbitrary 3D skeleton motion is required, making it directly compatible with the established 3D animation pipeline. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing, including dynamics, and a neural generative dynamic texture model creates corresponding dynamic texture maps. We show that by merely providing new skeletal motions, our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches, and even in real-time.

Gait and Full Body Movement Dataset of Autistic Children Classified by Rough Set Classifier

Gait and body movement are a window to human brain which make these activities unique for each person. These activities are used to diagnose some disorders related to parts of brain which causes have not been known such as Autism Disorders (AD). The traditional diagnostic methods of AD are time-consuming and highly dependent on clinician’s judgment which is based on behaviour assessment. This approach leads to subjective interpretations that differ from doctor to another and affect by strengths and weaknesses of patient. Therefore this paper aims to diagnosis of AD based on gait and body movement analysis. At first, Kinect v2 uses to create a 3D dataset, which includes three dimensional joints positions, joints trajectories video, skeleton movement video captured by Kinect v2, and color videos captured by Samsung Note 9 camera. This paper also aims to classify children with autism from normal children by proposed system based on four stages: Augmentation of the database by using seven transformations to deal with small number of autism cases; Extracting features that we think play an important role in classification; Reducing data dimensions using Principal Component Analysis; using Rough Set to classify dataset. Results show that classification is 92% after augmentation.

Design of a fluid-driven 3D printed spinal posture corrector

Abstract Application of 3D printing materials is increasing in the recent bio-medical engineering applications. Neglectful attitude toward body posture could bring the formation of spinal diseases. Improver posture during the working process induces the pain at the lower back and pectoral kyphosis. Disregard of treatment at the initial stage will further deteriorate the spinal disorder and force surgical intervention. The paper discussed several important factors together with the 3D printed spine. Thereby, this paper purpose corrector device, which intended to adjust the posture of the spine in daily life and form proper posture habits for the user. The device is equipped with a flex sensor, which detects forward spinal bending, whereas the IMU sensor measures the side motion of the body. The novel fluid-driven mechanism carries out the motion of the spinal skeleton. The origami-inspired skeleton is capable of imitating the movement of the real backbone. Under negative pressure, the skeleton constructs correct spine curvature. The algorithm inside the microcontroller processes data from the sensor and controls the actuator. A simple manufacturing process makes the design suitable for customized production.

Estimation of Finite Finger Joint Centers of Rotation Using 3D Hand Skeleton Motions Reconstructed from CT Scans

The present study proposed a method to estimate the finite finger joint centers of rotation (CoRs) with high accuracy using 3D hand skeleton motions reconstructed from CT scans. Ten hand postures starting from a fully extended posture and ending at a fist posture with about 10° difference in flexion between the adjacent postures were captured by a CT scanner for 15 male participants, and their 3D hand skeletons were reconstructed using the CT scans. Each bone segment from the full extension posture was registered to the corresponding bone segments of the remaining hand postures. The proximal bone segments of a joint from two postures were aligned to estimate the finite CoR of the joint between the two postures. Centerlines of the distal bone segments of the joint were then identified using the principal component analysis method, and the finite CoR of the joint was determined as the intersection point of the identified centerlines. The proposed method reduced the variation of estimated finite joint CoRs by 16.0% to 67.0% among the finger joints compared to the existing methods. The variation of estimated finite joint CoRs decreased as the rotation angle of the joint increased. The proposed method can be used for the simulation of finger movement with high accuracy.

RETRACTED: Image recognition of martial arts movements based on FPGA and image processing

The work is to perceive the intricate karate activity groupings consequently. It is to give a proportion of the nature of the activity performed. This is because there is a problem that requires essentially model. This work can be removed from the two motion capture hardware and consumer derived from the input sequence. The solution for the skeleton's movement proposed a depth-sensing system. The speed of the body's action because there is a karate sport classification.On the other hand, because the practice with even skilled martial arts for years, behavior patterns are repetitive. Similarly, there is no interest in the commercial market for solutions used in today's computer entertainment and tutoring system. The future of collecting analysis of elements present in the correct posture, accelerated by a punch and kick, the different aspects of performance, such as synchronizing the limb's movement and evaluation. This work aims to provide a survey to evaluate the quality of the whole body movement technology. It proposes a data set and its plan to implement the first reference file of the work's future. Generally speaking, FPGA is programmable with a group of programmable interconnect resources through digital circuits or programmable logic blocks surrounded by input/output blocks. Any system silicon chip is put together. The processor and the FPGA are essentially indeed parallel.

High-Detail Animation of Human Body Shape and Pose From High-Resolution 4D Scans Using Iterative Closest Point and Shape Maps

In this article, we present a method of analysis for 3D scanning sequences of human bodies in motion that allows us to obtain a computer animation of a virtual character containing both skeleton motion and high-detail deformations of the body surface geometry, resulting from muscle activity, the dynamics of the motion, and tissue inertia. The developed algorithm operates on a sequence of 3D scans with high spatial and temporal resolution. The presented method can be applied to scans in the form of both triangle meshes and 3D point clouds. One of the contributions of this work is the use of the Iterative Closest Point algorithm with motion constraints for pose tracking, which has been problematic so far. We also introduce shape maps as a tool to represent local body segment deformations. An important feature of our method is the possibility to change the topology and resolution of the output mesh and the topology of the animation skeleton in individual sequences, without requiring time-consuming retraining of the model. Compared to the state-of-the-art Skinned Multi-Person Linear (SMPL) method, the proposed algorithm yields almost twofold better accuracy in shape mapping.

Human Body Three-Dimensional Motion Skeleton Reconstruction of Moving Image Sequence

In the field of computer vision, the research of human motion analysis has a wide range of application prospects. Due to the complexity of human motion, the existing research methods have imposed many restrictions on the human body of the research object. A new method is proposed in this paper. The information of various types of human motion mainly discusses the reconstruction part of the human body's three-dimensional motion skeleton. The basic idea is to establish the image of each image based on the camera calibration and the application of three-dimensional human body model knowledge and motion continuity. On the basis of obtaining the binarized motion image, automatic annotation of joint feature points is realized. Firstly, the Canny operator is used to extract the target contour, and the Douglas-Peucker vector compression algorithm is used to obtain a suitable human contour shape. According to the joint proportional structure, the position of each joint point is determined, and the joint feature points are automatically labeled. The three-dimensional human motion tracking technology is studied. The characteristic optical flow algorithm is used to track the marked joint points, and the Kalman filter is used to correct the joint points of the tracking error. The automatic labeling method of the mannequin joint points used reduces the manual intervention in the system to some extent.

Learning Spatial-Temporal Representations Over Walking Tracklet for Long-Term Person Re-Identification in the Wild

Long-term person re-identification (re-ID) aims to build identity correspondence of the Target Subject of Interest (TSI) exposed under surveillance cameras over a long time interval. Compared to the conventional short-term re-ID studied by most existing works, it suffers an additional problem: significant dressing change observed with time lapsing. Unfortunately, this variation in long-term person re-ID case contradicts the assumption of prior short-term re-ID approaches, and thus causes significant difficulties if conventional short-term re-ID methods are applied. To address the problem, this paper proposes to learn hybrid feature representation via a two-stream network named SpTSkM, including a spatial-temporal stream and a skeleton motion stream. The former performs directly on image sequences, which tends to learn identity-related spatial-temporal patterns such as body geometric structure and body movement. The latter operates on normalized 3D skeletons by adapting graph convolutional network, which tends to learn pure motion patterns from skeleton sequences. Both streams extract fine-grained level time-gap stable information that is robust to appearance changes in long-term re-ID and meanwhile maintains sufficient discriminability to differentiate different people. The final matching metric is obtained by mixing information of the two streams in a score-level fusion strategy. In addition, we collect a Cloth-Varying vIDeo re-ID (CVID-reID) dataset particularly for long-term re-ID. It contains video tracklets of celebrities posted on the Internet. These videos are snapshots under extremely different scenarios that include highly dynamic background, diverse camera views and abundant cloth variations on each TSI. These factors cause CVID-reID more complicated and closer to practice. Our experiments demonstrate the difficulty of long-term person re-ID and also validate the effectiveness of the proposed SpTSkM, showing the best performance.

MotioNet

We introduce MotioNet , a deep neural network that directly reconstructs the motion of a 3D human skeleton from a monocular video. While previous methods rely on either rigging or inverse kinematics (IK) to associate a consistent skeleton with temporally coherent joint rotations, our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used motion representation. At the crux of our approach lies a deep neural network with embedded kinematic priors, which decomposes sequences of 2D joint positions into two separate attributes: a single, symmetric skeleton encoded by bone lengths, and a sequence of 3D joint rotations associated with global root positions and foot contact labels. These attributes are fed into an integrated forward kinematics (FK) layer that outputs 3D positions, which are compared to a ground truth. In addition, an adversarial loss is applied to the velocities of the recovered rotations to ensure that they lie on the manifold of natural joint rotations. The key advantage of our approach is that it learns to infer natural joint rotations directly from the training data rather than assuming an underlying model, or inferring them from joint positions using a data-agnostic IK solver. We show that enforcing a single consistent skeleton along with temporally coherent joint rotations constrains the solution space, leading to a more robust handling of self-occlusions and depth ambiguities.