Virtual Human Motion Research Articles

A language‐directed virtual human motion generation approach based on musculoskeletal models

AbstractThe development of the systems capable of synthesizing natural and life‐like motions for virtual characters has long been a central focus in computer animation. It needs to generate high‐quality motions for characters and provide users with a convenient and flexible interface for guiding character motions. In this work, we propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion, which lays the foundation for the development of language‐directed controllers in physics‐based character animation. First, we construct a simplified model of musculoskeletal dynamics for the virtual character. Subsequently, we propose a hierarchical control framework consisting of a trajectory tracking layer and a muscle control layer, obtaining the optimal control policy for imitating the reference motions through the training. We design a multi‐policy aggregation controller based on large language models, which selects the motion policy with the highest similarity to user text commands from the action‐caption data pool, facilitating natural language‐based control of virtual character motions. Experimental results demonstrate that the proposed approach not only generates high‐quality motions highly resembling reference motions but also enables users to effectively guide virtual characters to perform various motions via natural language instructions.

Improvement of Students’ Sports Performance in VR-Based Smart Sports Teaching in Colleges and Universities

Abstract In the complex landscape of educational information, the integration of Virtual Reality (VR) technology into secondary school physical education presents a promising avenue to enhance both teaching quality and efficiency. This article delineates the construction of a VR-based smart sports classroom, focusing on the overarching structure of the teaching program and the pivotal teaching processes. It discusses methods for analyzing and editing sports teaching videos and details the integration of VR with sports posture estimation. This includes the development of a virtual human body motion imaging model and corresponding motion-solving algorithms. An empirical study was conducted within a secondary school sports classroom to assess the feasibility and impact of VR technology on physical education. The study involved a comparative analysis of physical quality between an experimental group taught using VR technology and a control group taught using traditional methods. Additionally, the study examined changes in learning interest pre- and post-intervention in both groups. Statistical analysis revealed that the p-values for the three evaluated physical tests—20-meter sprint, shuttle run, and 30-second rope skipping—were all above 0.05, suggesting no significant difference in physical performance due to the VR intervention. However, significant improvements were observed in the dimensions of sports participation (p < 0.05), positive interest (p = 0.013 < 0.05), and focus on sports (p > 0.05), indicating an enhancement in students’ engagement and attitudes toward learning. Conversely, the negative interest dimension also showed significant changes (p < 0.05). These findings underscore the beneficial role of VR technology in enriching the learning experience and potentially improving academic outcomes in college-level physical education.

Muscle‐driven virtual human motion generation approach based on deep reinforcement learning

AbstractWe propose a muscle‐driven motion generation approach to realize virtual human motion with user interaction and higher fidelity, which can address the problem that the joint‐driven fails to reflect the motion process of the human body. First, a simplified virtual human musculoskeletal model is built based on human biomechanics. Then, a hierarchical policy learning framework is constructed including motion tracking layer, SPD controller and muscle control layer. The motion tracking layer is responsible for mimicking reference motion and completing control command, using proximal policy optimization to train the policy; the muscle control layer is aimed to minimize muscle energy consumption and train the policy based on supervised learning; the SPD controller acts as a link between the two layers. At the same time, we integrate the curriculum learning to improve the efficiency and success rate of policy training. Simulation experiments show that the proposed approach can use motion capture data and pose estimation data as reference motions to generate better and more adaptable motions. Furthermore, the virtual human has the ability to respond to the user control command during the motion, and can complete the target task successfully.

Research on Virtual Human Motion Control Based on Computer-Assisted Multimedia Simulation.

In order to reproduce realistic human motion in a virtual environment, a study of virtual human motion control based on computer-aided ergonomics simulation is proposed. First, we establish the required human body, workpiece, and production line simulation environment, and on this basis, we use OWAS, BSHA, and other analysis tools to carry out ergonomic simulation of virtual human motion simulation research, seek solutions to man-machine problems, and provide a new, efficient, and quantitative evaluation method for production line planning. The simulation results show that during the operation of virtual human 1, the ratio of the actual load on the left and right hands to the ultimate load has reached up to 70%. Therefore, it is necessary to revise the production line plan, and through analysis, it is concluded that the reason for the bad posture is due to the low height of the storage point and the machine tool. We believe that with continuous development, ProcessSimulate and ProcessDesigner will play a greater role in enterprise digitization.

Virtual Human Motion Extension Based on Bayesian Network

Aiming at the problem of virtual human motion reuse, the concept of motion extension is proposed, which includes lengthways extension and transverse extension. The lengthways extension extends motion along the time, and generates more frames from existing motion. The transverse extension extends motion to other characters, and multiple characters motion is generated by single character motion. Each motion in the group is similar and different from each other. Motion extension provides unified solution to important issues such as motion prediction, motion repair, and group motion. We construct the Bayesian network of the virtual human motion, which studies and reduces the dimension of the virtual human motion, and propose a method of motion extension based on Bayesian network. Experiments show that our method can generate more realistic virtual human motion based on existing motion, and provide support for the application of virtual human in the fields of bio-engineering, medicine and military.

Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning

There have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method, we build a hierarchical neuromuscular virtual human (NMVH) motion control system that consists of a low-level spine reflex layer and a high-level policy control layer. The spine reflex layer uses a feedback net to map sensory information to excitations, which stimulate muscles to generate joint torques. The policy control layer includes a deep neural network, which provides a learned action policy to spine reflex layer for achieving terrain-adaptive motion skills. The particle swarm optimization algorithm is used to optimize the gain factors of the feedback net for finding out a basic policy to make the virtual human walk on the flat terrain autonomously. The proximal policy optimization algorithm is employed to train the deep neural network in policy control layer for learning how to modulate the actions to adapt to the changing terrain. The simulation results in Matlab show that virtual human can walk smoothly and better adapt to the given terrain changes. It demonstrates that our control system improves the terrain-adaptive walking skill of the neuromuscular virtual human.

A semi-automatic approach to implement rapid non-immersive virtual maintenance simulation

PurposeCurrent virtual simulation platforms provide various tools to generate non-immersive simulation processes purposefully in different domains. The generated simulation processes are adopted for analysis, presentation, demonstration and verification. In the virtual maintenance domain, this intuitive and visual method has benefitted product maintainability design and improvement. Generating an ideal and reasonable non-immersive virtual maintenance simulation is always time-consuming because of the complicated human operations and logical relationships involved. This study aims to propose a semiautomatic approach to increase efficiency in non-immersive virtual maintenance simulation implementation.Design/methodology/approachThe methodology analyzes the general catalogs of common maintenance tasks and explores the corresponding secondary development approaches of simulation tools that can achieve motion simulation in virtual environments, by focusing on the diversity, complexity and uncertainty in non-immersive virtual simulation process generation. Afterward, a single virtual human motion can be generated by controlling the parameters and indices of the simulation tools. Subsequently, all of the generated single motions are connected logically to simulate the entire maintenance process.FindingsInstead of selecting various tools, such as that in a traditional method, the proposed methodology analyzes and integrates the necessary basic parameters considering the characteristics of virtual maintenance simulation for a target maintenance activity.Originality/valueThe user can control the predefined parameters to generate the simulation combining several other simple operations in virtual environments. Consequently, the methodology decreases simulation tool selection and logic consideration and increases efficiency to a certain extent in non-immersive virtual maintenance simulation generation.

Virtual Human Motion Real-time Rendering Technology

In this paper, we summarized the existing three-dimensional virtual human body model representation method with real-time and realistic requirements as a starting point, and proposed several virtual human motion rendering method to realize virtual human real-time drawing.

Simulation of body extension motion based on dynamics virtualization

With the further development of virtual human simulation technology and motion modeling, it is an important task to construct and simplify the virtual human geometry model. Aiming at the issues including large computational overhead, unnatural model generation gesture, change of kinematic chain structure and high complexity, this paper takes the body extension motion as the research object, uses the forward dynamics calculation method, conducts analysis of dynamic control structure of arm movement, and uses inverse kinematics algorithm to control leg movement. According to the forward dynamics algorithm to study the collision contact model of the foot and the ground, the virtual human could act more naturally and consistently in extension motion and the computational complexity could be effectively reduced.

EDITING VIRTUAL HUMAN MOTION TECHNIQUES WITH DYNAMIC MOTION SIMULATOR AND CONTROLLER

Modifying realistic virtual human movement has become a challenging task to the researcher for computer games and animation development. To achieve realistic virtual human, the character movement must have same motion like real human. Virtual human movement can be created by blending different sources such as motion capture, dynamic and kinematics simulation. Editing dynamic movement requires a great skill from animator and takes a long time to setup. This paper presents a new technique for editing virtual human motion state using dynamic motion control in the real time animation. The system approach based on active dynamic control by normalizes the trajectory of vector space position. This technique explores the perfect balance in dynamic motion controls for virtual human motion initial and final states. For that purpose, an enhancement of proportional-derivative controller will be used. This paper focuses on three main parts; virtual human hierarchy, motion editing techniques and motion dynamic control.

Virtual human hybrid control in virtual assembly and maintenance simulation

To improve virtual human motion simulation and generation efficiency in virtual assembly and maintenance (A&M) environment, based on a detailed analysis of virtual human control methods in A&M simulation, a virtual human hybrid control method is presented in this paper. In the proposed method, virtual human A&M motion is divided into three types: primitive motion (PM), fine motion (FM) and transitional motion (TM). For PM, a real operator can drive a virtual human in real time for A&M operation via virtual reality interactive devices; in this way, the PM can be quickly generated. As for FM and TM generation, their semantic-level parameterized models including engineering attributes are created; then, their generation processes are represented; in the generation processes, a virtual human posture automatic reasoning model is established to support auto-generating A&M motion. The approaches in the present study are validated by generating virtual human motions in partial assembly process of auto engine. Results show that the proposed method can improve A&M motion simulation efficiency and support the fast generation of realistic A&M motion.

Real-time control of 3D virtual human motion using a depth-sensing camera for agricultural machinery training

To recreate human movements in a virtual environment in real time, we propose a new method for real-time tracking of 3D virtual full-body motion using a depth-sensing camera. The method uses natural interaction and a non-contact mode. The 3D virtual environment was constructed using a 3D graphics engine and human joint data were calculated using images acquired from a Prime Sense depth-sensing camera. Then skeletal data for the human model in a skinned mesh animation were separated by improving the mesh modules using a 3D graphics engine. Finally, motion data from the depth sensor were combined with joint data for the human model to yield full-body control of a virtual human (VH). Experimental results show that the proposed method can drive VH full-body movements in real time based on motion-sensing data. The method was applied in virtual driving training for agricultural machinery. Trainees can become familiar with the basic operations required for driving agricultural machinery using full-body motion instead of a mouse and keyboard. The training system is inexpensive and has high safety and a strong sense of immersion.

Researching on Virtual Human's Motion Retargeting Techniques Towards Heterogeneous Topological Structure

This survey firstly reviews motion capture technology and the current challenge associated with the application in virtual human's control area. Secondly simplifies Jack's joint hierarchical model in order to drive Jack's virtual human to move by motion capture data. Thirdly uses Euler angle rotation equation to achieve pose mapping between BVH and Jack. Finally, results from experimentation show that when the motion capture data contained rich human motion details are retargeted to Jack by the paper's algorithms, the real-time interaction between virtual human and environment seems more real.

Continuous Full-Body Motion Control of Virtual Human Using Sparse Wiimotes

Sparse sensors that recognize full-body human motion and that control the motion of virtual humans have emerged as valuable research tools in the field of human–computer interactions. Here we propose a method for motion recognition and prolonged, continuous generation of motion data based on the recognition results. The only inputs required are the directional accelerations collected by four Wii remotes, which are attached on the four limbs of a human. The extended and continuous signal sequences are separated into small segments that can be described by particular motion content. Use of a fused hidden Markov model (FHMM) during the recognition process ensures the accuracy and efficiency with which independent motion segments are recognized. A graph model enhances the capacity of classification when dealing with a signal sequence associated with a prolonged motion. During the reconstruction and generation processes, an efficient state-based motion graph generates the extended and continuous virtual human motion data, which accurately reflects variation in the movement of the actors. Our method has a strong capacity to classify types of motion upon their recognition and the control process can be applied to a range of applications involving interaction.

Overview of Virtual Reality Apply to Sports

Summarized sports simulation systems based virtual reality. The function include construction of virtual training scenarios, sports data capture, physiological and psychological data acquisition, action reproduce and training effectiveness icon analysis. The key technologies include physical and physiological modeling, virtual human animation, motion capture and real-time rendering and interaction. In the future, 3D simulation of technique action reproduce will continually be the hot point, need to cultivate talents both accomplished in sports and computer sciences.

Dual exponential map parameterization of maintenance Agent model

Motion parameterization of complex joint model is the foundation of virtual human motion simulation. The classical parameterization methods of 3D rotation,such as rotation matrix,Euler angle,and quaternion,which bear several drawbacks,cannot resolve the motion parameterization of complex joint model. Based on the improvement of exponential map,a novel motion parameterization method called Dual Exponential Map (DEM) which can depict translation and rotation,linear velocity and angular velocity simultaneously was proposed. The motion description as well as parameters computation capabilities of the DEM were introduced. Finally a simulation on maintenance operation training was implemented. The experimental results illustrate that the DEM method bears powerful capability on derivation,solution to differential and ODEs,optimization control and interpolation.

A Human-Centered Virtual Assembly System

Virtual manufacturing technology has become an effective method for decision and planning in manufacturing. Due to ergonomics problems are widely concerned in assembly design planning, a human-centered virtual assembly system framework is proposed for ergonomics analysis for assembly operation in this paper. The six-layer framework integrates virtual human modeling, motion capture and recognition, ergonomics evaluation and virtual assembly process planning as an organic whole. Data exchanging and system function are discussed based on this framework. The work in virtual reality (VR) technology, motion capture technology, ergonomics method and optimization method for implementing the system is also described. The framework would provide a new approach for the combination of virtual manufacturing and ergonomics analysis in the future.

Modified virtual build methodology for computer‐aided ergonomics and safety

AbstractIn the past, computer‐based ergonomic analysis has been time‐consuming and has required many assumptions when the virtual human motion paths have been programmed. Hence, it appears that many in the human factors and ergonomics profession have not developed much expertise in the Digital Human Modeling area. The Virtual Build method first demonstrated by the Ford Motor Company appears to improve the accuracy and speed of the ergonomic analyses used in design of assembly processes. It allows capture of real‐motion and virtual process interaction rather than assuming expected posture and task motion. A variation of the Virtual Build methodology, presented in this article, affords the benefit that both product and process design can be analyzed prior to production and limitations have been overcome. Hardware limitations in the Virtual Build environment previously prevented transfer of the data collection and analysis methods to the real manufacturing environment and limited the ability to validate the predictions from the virtual industrial environment. Preliminary results of the case illustrating the modified Virtual Build research methodology presented in this article appear to be promising in support of computer‐aided ergonomics and safety for design of new or existing workstations and consumer products. A mixed‐mode virtual‐real environment that provides force feedback can be used to supplement the experimental setup and improve fidelity of the interactions in a work design assessment. A summary of the comfort assessment for a product design example that was tested with constrained human motion is outlined in the results section and presented in the context of a summary of key issues in Virtual Interactive Design and Virtual Build Assessments. © 2007 Wiley Periodicals, Inc. Hum Factors Man 17: 413–422, 2007.

Match the Virtual Human Model and Motion Data Automatically

Captured motion data is widely used in virtual human motion control and synthesis. Usually, the motion data has a native skeleton definition. To apply captured motion on virtual human skin model, the model should have an underlying skeleton that matches the one defined by the motion data. This paper proposes an algorithm called LMSA (lazy match based on semantic analysis) which generates skeleton for existing human model and matches it to the motion data when the motion data is loaded. The LMSA algorithm first generates Candidate-Joint-Set for a human model with a group of parallel planes and then applies the same semantic analysis to both the Candidate-Joint-Set and the skeleton of motion data to match them. By using LMSA algorithm, different motion data can be applied to the existing human model directly without predefining skeleton for human model.