3D Human Motion Data Research Articles

FLK: A filter with learned kinematics for real-time 3D human pose estimation

There is a growing interest in adopting 3D human pose estimation in safety-critical systems, from healthcare to Industry 5.0. Nevertheless, when applied in such settings, these neural networks may suffer from estimation inaccuracy. Besides imprecise or inconsistent annotations in the training dataset, the inaccuracy is caused by poor image quality, rare poses, dropped frames, or heavy occlusions in the scene. In addition, these scenarios often require the software results to have temporal constraints, such as real-time and zero- or low-latency, which make many of the filtering solutions proposed in the literature inapplicable. This paper proposes FLK, a Filter with Learned Kinematics, to refine 3D human motion data in real-time and at zero/low latency. The temporal core combines a Kalman filter and a low-pass filter, which learns the motion model through a recurrent neural network. The spatial core takes advantage of the biomechanical constraints of the human body to provide spatial coherency between keypoints. The combination of the cores allows the filter to adequately address different types of noise, from jittering to dropped frames. We test the filter on motion data from multiple datasets and seven 3D human pose estimation backbones, improving accuracy up to 140 mm with non-Gaussian noise and 53 mm with missing information.

3D human pose data augmentation using Generative Adversarial Networks for robotic-assisted movement quality assessment.

In the realm of human motion recognition systems, the augmentation of 3D human pose data plays a pivotal role in enriching and enhancing the quality of original datasets through the generation of synthetic data. This augmentation is vital for addressing the current research gaps in diversity and complexity, particularly when dealing with rare or complex human movements. Our study introduces a groundbreaking approach employing Generative Adversarial Networks (GANs), coupled with Support Vector Machine (SVM) and DenseNet, further enhanced by robot-assisted technology to improve the precision and efficiency of data collection. The GANs in our model are responsible for generating highly realistic and diverse 3D human motion data, while SVM aids in the effective classification of this data. DenseNet is utilized for the extraction of key features, facilitating a comprehensive and integrated approach that significantly elevates both the data augmentation process and the model's ability to process and analyze complex human movements. The experimental outcomes underscore our model's exceptional performance in motion quality assessment, showcasing a substantial improvement over traditional methods in terms of classification accuracy and data processing efficiency. These results validate the effectiveness of our integrated network model, setting a solid foundation for future advancements in the field. Our research not only introduces innovative methodologies for 3D human pose data enhancement but also provides substantial technical support for practical applications across various domains, including sports science, rehabilitation medicine, and virtual reality. By combining advanced algorithmic strategies with robotic technologies, our work addresses key challenges in data augmentation and motion quality assessment, paving the way for new research and development opportunities in these critical areas.

A markless 3D human motion data acquisition method based on the binocular stereo vision and lightweight open pose algorithm

This study aimed to achieve precise 3D human motion data acquisition without the need for markers. Specifically, the proposed system utilized two affordable cameras to capture human motion from diverse angles. It developed a lightweight Open Pose algorithm to extract 2D human body joints from recorded videos. Subsequently, a stereo-matching method was applied to derive accurate 3D human body joint data. Comprehensive experiments with 10 healthy participants showcased the system's exceptional performance, demonstrating superior measurement accuracy compared to established standards like the NOKOV system (RMSE values consistently below 9°) and commonly used Microsoft Kinect devices (V2 and Azure). This innovative binocular vision system presents a new approach to motion capture, potentially offering a cost-effective and precise tool for practical applications across various scenarios (e.g., motor functional assessment, virtual avatar streaming), contributing significantly to both scientific and academic communities.

Contact Part Detection From 3D Human Motion Data Using Manually Labeled Contact Data and Deep Learning

Contact Part Detection From 3D Human Motion Data Using Manually Labeled Contact Data and Deep Learning

Hierarchical learning recurrent neural networks for 3D motion synthesis

Three-dimensional human motion synthesis is one of the key technologies in the field of computer animation and multimedia applications. It is well known that the human body's own motion is full of strong personality, emotion, and high-dimensional characteristics, leading to the automatic synthesis of diverse and lifelike 3D human motion data continues to be a challenging task. Facing the challenge, this paper proposes a human motion synthesis framework based on hierarchical learning recurrent neural networks (HL-RNN). The framework includes a low-level network and a high-level network, which are used to extract the path information of the movement and the spatio-temporal relationship of the human bone structure, respectively. Then, after fusion, motions that satisfy the path constraints could be generated. This method can not only synthesize high-quality human movements that follow a specified trajectory, but also synthesize smooth transitions between various movements, and can also be used to synthesize data of different motion styles. Compared with some latest methods, experiments showed that the proposed method can significantly improve the quality and generalization performance of motion synthesis.

3D Human Motion Synthesis Based on Convolutional Neural Network

Human motion synthesis technology has a very important position in computer animation, and it is widely used in medicine, film and television, motion analysis, games, and other related fields. The synthesis of human motion is the virtual of the action of the characters in the real world, the authenticity of the action, and the natural smoothness is especially important to the user's experience. Due to the complexity of human structure, how to generate a high-quality movement is a challenging task. The data used in this paper are all 3D human motion data in BioVision Hierarchical (BVH) format, which can be captured by optical, inertial, mechanical or other video-based motion capture devices. In this paper, first, a three-layer convolutional neural network was used to output mapping in the hidden unit of the input motion capture data. Then, a one-dimensional convolution auto-encoder was connected; meanwhile, the bone length constraint, position constraint, and trajectory constraint were added. It repaired the non-inertial joints of motion data and removed the motion artifacts. To achieve the synthesis of the two motions, we extracted the style transformation in the motion, added style and content constraints, and finally output the motion. To verify the feasibility of the algorithm, we obtained the animation effect of the synthesized motion by testing the input motion. The experimental results show that the motions synthesized by the proposed algorithm not only look natural smooth in visual effect but also reduce the time consumed by about 42.6% compared with the existing algorithms.

Gesture imitation and recognition using Kinect sensor and extreme learning machines

Abstract This study presents a framework that recognizes and imitates human upper-body motions in real time. The framework consists of two parts. In the first part, a transformation algorithm is applied to 3D human motion data captured by a Kinect. The data are then converted into the robot’s joint angles by the algorithm. The human upper-body motions are successfully imitated by the NAO humanoid robot in real time. In the second part, the human action recognition algorithm is implemented for upper-body gestures. A human action dataset is also created for the upper-body movements. Each action is performed 10 times by twenty-four users. The collected joint angles are divided into six action classes. Extreme Learning Machines (ELMs) are used to classify the human actions. Additionally, the Feed-Forward Neural Networks (FNNs) and K-Nearest Neighbor (K-NN) classifiers are used for comparison. According to the comparative results, ELMs produce a good human action recognition performance.

Multichannel Human Motion Similarity Analysis Based on Information Entropy and Dynamic Time Warping

Evaluating motion similarity between trainer and trainee is a key part in computer-assisted sports teaching system. Our similarity evaluation algorithm mainly contains four steps. Firstly, the multichannel 3D human motion data are captured using the Kinect, a depth sensor of Microsoft. Next, in order to greatly reduce the amount of data analysis, the piecewise extremum method (PEM) is applied to achieve this goal. Then, considering that doing the same motions the rhythms of different people are not synchronized, the Dynamic Time Warping algorithm (DTW) is selected to solve the problem of analyzing one channel unequal length motion sequences. Finally, the similarity between the two sets of multichannel human motion sequences can be evaluated using the combined method of the information entropy and DTW. The experimental results indicate that compared with other traditional methods, the proposed method not only accurately measures similarity degree of different motions, but also requires less computational time and memory storage capacity.