Real Human Motion Research Articles

Inertial Motion Capture-Driven Digital Human for Ergonomic Validation: A Case Study of Core Drilling.

In the evolving realm of ergonomics, there is a growing demand for enhanced comfortability, visibility, and accessibility in the operation of engineering machinery. This study introduces an innovative approach to assess the ergonomics of a driller's cabin by utilizing a digital human. Through the utilization of inertial motion capture sensors, the method enables the operation of a virtual driller animated by real human movements, thereby producing more precise and realistic human-machine interaction data. Additionally, this study develops a simplified model for the human upper limbs, facilitating the calculation of joint forces and torques. An ergonomic analysis platform, encompassing a virtual driller's cabin and a digital human model, is constructed using Unity 3D. This platform enables the quantitative evaluation of comfortability, visibility, and accessibility. Its versatility extends beyond the current scope, offering substantial support for product development and enhancement.

What Makes a Movement Human‐Like?

AbstractWith the advancement of AI‐generated human motion, it is of increasing importance to think about how we distinguish real human motion from machine‐generated movements. In this study, we recruited professional performers to use the whole body to make a short movement to inform potential observers that they are real humans (instead of machines). Their movements were captured with a motion capture system (Vicon) and later reduced to dynamic point‐like displays (biological motion). They were interviewed after the recording to provide their acting strategies. Naive observers who did not participate in the motion data collection were recruited to watch these videos and judge whether the biological motions looked human‐like or not (YES/NO), as well as to report their judging criteria. The major factors extracted from these reports include kinematics, context, body mechanics, and principles of physical laws. We discuss the impact of these criteria and how they may possibly help improve the future generation of human‐like motions.

3D human pose data augmentation using Generative Adversarial Networks for robotic-assisted movement quality assessment.

In the realm of human motion recognition systems, the augmentation of 3D human pose data plays a pivotal role in enriching and enhancing the quality of original datasets through the generation of synthetic data. This augmentation is vital for addressing the current research gaps in diversity and complexity, particularly when dealing with rare or complex human movements. Our study introduces a groundbreaking approach employing Generative Adversarial Networks (GANs), coupled with Support Vector Machine (SVM) and DenseNet, further enhanced by robot-assisted technology to improve the precision and efficiency of data collection. The GANs in our model are responsible for generating highly realistic and diverse 3D human motion data, while SVM aids in the effective classification of this data. DenseNet is utilized for the extraction of key features, facilitating a comprehensive and integrated approach that significantly elevates both the data augmentation process and the model's ability to process and analyze complex human movements. The experimental outcomes underscore our model's exceptional performance in motion quality assessment, showcasing a substantial improvement over traditional methods in terms of classification accuracy and data processing efficiency. These results validate the effectiveness of our integrated network model, setting a solid foundation for future advancements in the field. Our research not only introduces innovative methodologies for 3D human pose data enhancement but also provides substantial technical support for practical applications across various domains, including sports science, rehabilitation medicine, and virtual reality. By combining advanced algorithmic strategies with robotic technologies, our work addresses key challenges in data augmentation and motion quality assessment, paving the way for new research and development opportunities in these critical areas.

AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion

Generating realistic human motion sequences from text descriptions is a challenging task that requires capturing the rich expressiveness of both natural language and human motion. Recent advances in diffusion models have enabled significant progress in human motion synthesis. However, existing methods struggle to handle text inputs that describe complex or long motions. In this paper, we propose the Adaptable Motion Diffusion (AMD) model, which leverages a Large Language Model (LLM) to parse the input text into a sequence of concise and interpretable anatomical scripts that correspond to the target motion. This process exploits the LLM’s ability to provide anatomical guidance for complex motion synthesis. We then devise a two-branch fusion scheme that balances the influence of the input text and the anatomical scripts on the inverse diffusion process, which adaptively ensures the semantic fidelity and diversity of the synthesized motion. Our method can effectively handle texts with complex or long motion descriptions, where existing methods often fail. Experiments on datasets with relatively more complex motions, such as CLCD1 and CLCD2, demonstrate that our AMD significantly outperforms existing state-of-the-art models.

MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators

Generating realistic human motion from given action descriptions has experienced significant advancements because of the emerging requirement of digital humans. While recent works have achieved impressive results in generating motion directly from textual action descriptions, they often support only a single modality of the control signal, which limits their application in the real digital human industry. This paper presents a Motion General-Purpose generaTor (MotionGPT) that can use multimodal control signals, e.g., text and single-frame poses, for generating consecutive human motions by treating multimodal signals as special input tokens in large language models (LLMs). Specifically, we first quantize multimodal control signals into discrete codes and then formulate them in a unified prompt instruction to ask the LLMs to generate the motion answer. Our MotionGPT demonstrates a unified human motion generation model with multimodal control signals by tuning a mere 0.4% of LLM parameters. To the best of our knowledge, MotionGPT is the first method to generate human motion by multimodal control signals, which we hope can shed light on this new direction. Visit our webpage at https://qiqiapink.github.io/MotionGPT/.

Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective

Estimating 3D full-body pose from sparse sensor data is a pivotal technique employed for the reconstruction of realistic human motions in Augmented Reality and Virtual Reality. However, translating sparse sensor signals into comprehensive human motion remains a challenge since the sparsely distributed sensors in common VR systems fail to capture the motion of full human body. In this paper, we use well-designed Body Pose Graph (BPG) to represent the human body and translate the challenge into a prediction problem of graph missing nodes. Then, we propose a novel full-body motion reconstruction framework based on BPG. To establish BPG, nodes are initially endowed with features extracted from sparse sensor signals. Features from identifiable joint nodes across diverse sensors are amalgamated and processed from both temporal and spatial perspectives. Temporal dynamics are captured using the Temporal Pyramid Structure, while spatial relations in joint movements inform the spatial attributes. The resultant features serve as the foundational elements of the BPG nodes. To further refine the BPG, node features are updated through a graph neural network that incorporates edge reflecting varying joint relations. Our method's effectiveness is evidenced by the attained state-of-the-art performance, particularly in lower body motion, outperforming other baseline methods. Additionally, an ablation study validates the efficacy of each module in our proposed framework.

Human-like acceleration and deceleration control of a robot astronaut floating in a space station

The acceleration and deceleration (AD) motions are the basic motion modes of robot astronauts moving in a space station. Controlling the locomotion of the robot astronaut is very challenging due to the strong nonlinearity of its complex multi-body dynamics in a gravity-free environment. However, after training, humans can move well in space stations by pushing the bulkhead, and the motion mechanism of humans is a good reference for robot astronauts. The contribution of this study is modeling the human AD motion in a microgravity environment and proposing a human-like control method for robot astronauts moving in space stations. Specifically, the movement and contact force data of the human body during AD motion were collected on an air-floating platform. Through human AD modeling analysis, the mechanism of human motion is discovered, and semi-sinusoidal primitives of contact forces are proposed for AD motion. Then, a dynamic guidance model of human AD motion is built to complete motion planning under contact constraints, which is used as the expected model for the AD control of robot astronauts. Benefiting from the force primitives, accurate and safe planning of human-like AD motion can be completed. The characteristics and mechanism of human AD motion have been analyzed from the perspective of optimization. Lastly, based on the proposed dynamic guidance model, the AD motion policy is mapped to the robot astronaut system via a system control method based on the equivalent mapping of dynamic responses (force, velocity and pose). Through comparative analysis with real human motion data and simulation results under different conditions, the proposed AD control method can achieve human-like motion efficiently and stably. Even when confronted with errors in the robot’s contact velocities and inertia parameters, the method can significantly reduce the motion errors while ensuring stability.

A study on contaminant leakage from Airborne Infection Isolation room during medical staff entry; Implementation of walking motion on hypothetical human model in CFD simulation

The recent outbreak of respiratory infectious diseases such as Middle East Respiratory Syndrome (MERS) and Coronavirus Disease-19 (COVID-19) highlights the critical role of Air Infection Isolation (AII) rooms in preventing the spread of disease. However, the risk of contaminant leakage from the AII room during medical staff interactions remains a significant challenge. This study advances our understanding by employing Computational Fluid Dynamics (CFD) simulations with a novel approach. While CFD simulations mitigate some limitations of field experiments, such as measurement resolution and variable changes, their accuracy is often limited by the oversimplification of models, particularly in representing human models and movement. Unlike prior studies, we incorporate minimal simplification, featuring a human model with Personal Protective Equipment (PPE) and realistic walking motions. This study explores four distinct scenarios with varied human movements, offering a comprehensive analysis of airflow and contaminant dispersion in AII rooms. The results indicated significant differences in airflow patterns and contaminant distribution when realistic walking motion was implemented in the human model, as opposed to simplified movement models. The contaminant leakage from the AII room ranged from 20.6 % to 28.6 %, suggesting that prior studies may underestimate infection risks due to oversimplified human models and motions. These findings underscore the importance of implementing realistic human models and motions in CFD simulations for accurate infection risk assessment in AII rooms. Additionally, our study opens pathways for future research to explore more complex human interactions in clinical settings, enhancing infection control strategies.

Bridging Actions: Generate 3D Poses and Shapes In-Between Photos.

Generating realistic 3D human motion has been a fundamental goal of the game/animation industry. This work presents a novel transition generation technique that can bridge the actions of people in the foreground by generating 3D poses and shapes in-between photos, allowing 3D animators/novice users to easily create/edit 3D motions. To achieve this, we propose an adaptive motion network (ADAM-Net) that effectively learns human motion from masked action sequences to generate kinematically compliant 3D poses and shapes in-between given temporally-sparse photos. Three core learning designs underpin ADAM-Net. First, we introduce a random masking process that randomly masks images from an action sequence and fills masked regions in latent space by interpolation of unmasked images to simulate various transitions under given temporally-sparse photos. Second, we propose a long-range adaptive motion (L-ADAM) attention module that leverages visual cues observed from human motion to adaptively recalibrate the range that needs attention in a sequence, along with a multi-head cross-attention. Third, we develop a short-range adaptive motion (S-ADAM) attention module that weightedly selects and integrates adjacent feature representations at different levels to strengthen temporal correlation. By coupling these designs, the results demonstrate that ADAM-Net excels not only in generating 3D poses and shapes in-between photos, but also in classic 3D human pose and shape estimation.

Toward Realistic 3D Human Motion Prediction With a Spatio-Temporal Cross- Transformer Approach

Human motion prediction intends to predict how humans move given a historical sequence of 3D human motions. Recent transformer-based methods have attracted increasing attentions and demonstrated their promising performance in 3D human motion prediction. However, existing methods generally decompose the input of human motion information into spatial and temporal branches in a separate way and seldom consider their inherent coherence between the two branches, hence often failing to register the dynamic spatio-temporal information during the training process. Motivated by these issues, we propose a spatio-temporal cross-transformer network (STCT) for 3D human motion predictions. Specifically, we investigate various types of interaction methods (i.e., Concatenation Interaction, Msg token interaction, and Cross-transformer) to capture the coherence of the spatial and temporal branches. According to the obtained results, the proposed cross-transformer interaction method shows its superiority over other methods. Meanwhile, considering that most existing works treat the human body as a set of 3D human joint positions, the predicted human joints are proportionally less appropriate to the realistic human body due to unreasonable bone length and non-plausible poses as time progresses. We further resort to the bone constraints of human mesh to produce more realistic human motions. By fitting a parametric body model (i.e., SMPL-X model) to the predicted human joints, a reconstruction loss function is proposed to remedy the unreasonable bone length and pose errors. Comprehensive experiments on AMASS and Human3.6M datasets have demonstrated that our method achieves superior performance over compared methods.

Too Stiff, Too Strong, Too Smart

Deep reinforcement learning (DRL) methods have demonstrated impressive results for skilled motion synthesis of physically based characters, and while these methods perform well in terms of tracking reference motions or achieving complex tasks, several concerns arise when evaluating the naturalness of the motion. In this paper, we conduct a preliminary study of specific quantitative metrics for measuring the naturalness of motion produced by DRL control policies beyond their visual appearance. Namely, we propose to study the stiffness of the control policy, in anticipation that it will influence how the character behaves in the presence of external perturbation. Second, we establish two baselines for strength that allow evaluating the use of joint torques in comparison to human performance. Third, we propose the study of variability to reveal the unnatural precision of control policies and how they compare to real human motion. In sum, we aim to establish repeatable measures to assess the naturalness of control policies produced by DRL methods, and we present a set of comparisons from state-of-the-art systems. Finally, we propose simple modifications to improve realism on these axes.

GGTr: An Innovative Framework for Accurate and Realistic Human Motion Prediction

Human motion prediction involves forecasting future movements based on past observations, which is a complex task due to the inherent spatial-temporal dynamics of human motion. In this paper, we introduced a novel framework, GGTr, which adeptly encapsulates these patterns by integrating positional graph convolutional network (GCN) layers, gated recurrent unit (GRU) network layers, and transformer layers. The proposed model utilizes an enhanced GCN layer equipped with a positional representation to aggregate information from body joints more effectively. To address temporal dependencies, we strategically combined GRU and transformer layers, enabling the model to capture both local and global temporal dependencies across body joints. Through extensive experiments conducted on Human3.6M and CMU-MoCap datasets, we demonstrated the superior performance of our proposed model. Notably, our framework shows significant improvements in predicting long-term movements, outperforming state-of-the-art methods substantially.

Pandemic Simulator: An Agent-Based Framework with Human Behavior Modeling for Pandemic-Impact Assessment to Build Sustainable Communities

It is crucial to immediately curb the spread of a disease once an outbreak is identified in a pandemic. An agent-based simulator will enable policymakers to evaluate the effectiveness of different hypothetical strategies and policies with a higher level of granularity. This will allow them to identify vulnerabilities and asses the threat level more effectively, which in turn can be used to build resilience within the community against a pandemic. This study proposes a PanDemic SIMulator (PDSIM), which is capable of modeling complex environments while simulating realistic human motion patterns. The ability of the PDSIM to track the infection propagation patterns, contact paths, places visited, characteristics of people, vaccination, and testing information of the population allows the user to check the efficacy of different containment strategies and testing protocols. The results obtained based on the case studies of COVID-19 are used to validate the proposed model. However, they are highly extendable to all pandemics in general, enabling robust planning for more sustainable communities.

Activity triangles: a new approach to measure activity spaces

There is an on-going challenge to describe, analyse and visualise the actual and potential extent of human spatial behaviour. The concept of an activity space has been used to examine how people interact with their environment and how the actual or potential spatial extent of individual spatial behaviour can be defined. In this paper, we introduce a new method for measuring activity spaces. We first focus on the definitions and the applications of activity space measures, identifying their respective limitations. We then present our new method, which is based on the theoretical concept of significant locations, that is, places where people spent most of their time. We identify locations of significant places from GPS trajectories and define the activity space of an individual as a set of the first three significant places forming a so-called “activity triangle”. Our new method links the distances travelled for different activities to whether or not people group their activities, which is not possible using existing methods of measuring activity spaces. We test our method on data from a GPS-based travel survey across three towns is Scotland and look at the variations in size and shape of the designed activity triangle among people of different age and gender. We also compare our activity triangle with five other activity spaces and conclude by providing possible routes for improvement of activity space measures when using real human movement data (GPS data).

An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural Networks

We present an end-to-end method for capturing the dynamics of 3D human characters and translating them for synthesizing new, visually-realistic motion sequences. Conventional methods employ sophisticated, but generic, control approaches for driving the joints of articulated characters, paying little attention to the distinct dynamics of human joint movements. In contrast, our approach attempts to synthesize human-like joint movements by exploiting a biologically-plausible, compact network of spiking neurons that drive joint control in primates and rodents. We adapt the controller architecture by introducing learnable components and propose an evolutionary algorithm for training the spiking neural network architectures and capturing diverse joint dynamics. Our method requires only a few samples for capturing the dynamic properties of a joint's motion and exploits the biologically-inspired, trained controller for its reconstruction. More importantly, it can transfer the captured dynamics to new visually-plausible motion sequences. To enable user-dependent tailoring of the resulting motion sequences, we develop an interactive framework that allows for editing and real-time visualization of the controlled 3D character. We also demonstrate the applicability of our method to real human motion capture data by learning the hand joint dynamics from a gesture dataset and using our framework to reconstruct the gestures with our 3D animated character. The compact architecture of our joint controller emerging from its biologically-realistic design, and the inherent capacity of our evolutionary learning algorithm for parallelization, suggest that our approach could provide an efficient and scalable alternative for synthesizing 3D character animations with diverse and visually-realistic motion dynamics.

A Probabilistic Model of Human Activity Recognition with Loose Clothing.

Human activity recognition has become an attractive research area with the development of on-body wearable sensing technology. Textiles-based sensors have recently been used for activity recognition. With the latest electronic textile technology, sensors can be incorporated into garments so that users can enjoy long-term human motion recording worn comfortably. However, recent empirical findings suggest, surprisingly, that clothing-attached sensors can actually achieve higher activity recognition accuracy than rigid-attached sensors, particularly when predicting from short time windows. This work presents a probabilistic model that explains improved responsiveness and accuracy with fabric sensing from the increased statistical distance between movements recorded. The accuracy of the comfortable fabric-attached sensor can be increased by 67% more than rigid-attached sensors when the window size is 0.5s. Simulated and real human motion capture experiments with several participants confirm the model's predictions, demonstrating that this counterintuitive effect is accurately captured.

Research on 3D Human Motion Capture Algorithm for Online Physical Education Teaching

This study proposes to use the gradient descent method in data fusion to solve the problem of low learning efficiency of online sports courses by optimizing the attitude calculation and combining gait detection and zero speed correction to form a three-dimensional human motion capture system. The system was applied to online physical education teaching to improve teaching quality. The performance of the optimized attitude calculation algorithm and the Kalman filter algorithm was compared. The results show that the error angle of the data-fusion-attitude calculation algorithm was 1.67°, which is better than the Kalman filter algorithm. In the empirical analysis of the 3D human motion capture system designed by the research, the system could correctly capture the real human motion trajectory. Moreover, comparative analysis of the online physical education courses showed that the satisfaction of the online physical education courses embedded in the system was 90.62, which was much higher than other online physical education courses. These results show that the optimized attitude calculation algorithm can better capture real human movements, and the human motion capture system composed of this can improve the learning effect of online physical education courses. This method opens up new ideas for online physical education teaching.

Human Motion Transfer With 3D Constraints and Detail Enhancement.

We propose a new method for realistic human motion transfer using a generative adversarial network (GAN), which generates a motion video of a target character imitating actions of a source character, while maintaining high authenticity of the generated results. We tackle the problem by decoupling and recombining the posture information and appearance information of both the source and target characters. The innovation of our approach lies in the use of the projection of a reconstructed 3D human model as the condition of GAN to better maintain the structural integrity of transfer results in different poses. We further introduce a detail enhancement net to enhance the details of transfer results by exploiting the details in real source frames. Extensive experiments show that our approach yields better results both qualitatively and quantitatively than the state-of-the-art methods.

A human motion angle prediction method using a neural network and feature-extraction-processed sEMG obtained in real time to utilize mechanical delay of sEMG

Surface electromyogram (sEMG) has been used to estimate human motions. Feature extraction (FE) has been proposed as a process to decrease noise and extract information about muscle activities from measured raw sEMG signals. Electromechanical delay (EMD) is a time delay between a generation of sEMG and subsequent occurrence of human motions, and sEMG can be obtained in advance of motions occurrence by using this time delay. The purpose of this study is to propose a real time human motion prediction method using a neural network model and FE-processed sEMG signals utilizing EMD. Experimental results show that the proposed method has better prediction results among other methods.

Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi‐Hot Class Embedding

AbstractSynthesizing multi‐character interactions is a challenging task due to the complex and varied interactions between the characters. In particular, precise spatiotemporal alignment between characters is required in generating close interactions such as dancing and fighting. Existing work in generating multi‐character interactions focuses on generating a single type of reactive motion for a given sequence which results in a lack of variety of the resultant motions. In this paper, we propose a novel way to create realistic human reactive motions which are not presented in the given dataset by mixing and matching different types of close interactions. We propose a Conditional Hierarchical Generative Adversarial Network with Multi‐Hot Class Embedding to generate the Mix and Match reactive motions of the follower from a given motion sequence of the leader. Experiments are conducted on both noisy (depth‐based) and high‐quality (MoCap‐based) interaction datasets. The quantitative and qualitative results show that our approach outperforms the state‐of‐the‐art methods on the given datasets. We also provide an augmented dataset with realistic reactive motions to stimulate future research in this area.