Human Motion Model Research Articles

Reliability Analysis of Multi-Objective Spatio-Temporal Segmentation of Human Motion in Video Sequences

In view of the problem of uneven distribution of edge contour of multi-target human motion image in video sequence, which leads to the decline of target detection ability, an algorithm of multi-target spatial-temporal segmentation of human motion in video sequence based on edge contour feature detection and block fusion is proposed. Firstly, a multi-target spatial-temporal detection model of human motion in video sequence was constructed, extracting video image frame sequence, using discrete frame fusion method to segment and fuse moving target image, matching moving multi-target in video sequence, secondly segmenting motion features in moving target image, combining with SURF algorithm (speeded up robust features, accelerated robust features) to detect and extract human motion objects in video sequence. The experimental results show that the gray histogram of human motion multi-target space-time segmentation is close to the original image histogram, and the detection and recognition ability of human motion target is improved.

Spatio-Temporal Manifold Learning for Human Motions via Long-Horizon Modeling.

Data-driven modeling of human motions is ubiquitous in computer graphics and computer vision applications, such as synthesizing realistic motions or recognizing actions. Recent research has shown that such problems can be approached by learning a natural motion manifold using deep learning on a large amount data, to address the shortcomings of traditional data-driven approaches. However, previous deep learning methods can be sub-optimal for two reasons. First, the skeletal information has not been fully utilized for feature extraction. Unlike images, it is difficult to define spatial proximity in skeletal motions in the way that deep networks can be applied for feature extraction. Second, motion is time-series data with strong multi-modal temporal correlations between frames. On the one hand, a frame could be followed by several candidate frames leading to different motions; on the other hand, long-range dependencies exist where a number of frames in the beginning are correlated with a number of frames later. Ineffective temporal modeling would either under-estimate the multi-modality and variance, resulting in featureless mean motion or over-estimate them resulting in jittery motions, which is a major source of visual artifacts. In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications. The network has a new spatial component for feature extraction. It is also equipped with a new batch prediction model that predicts a large number of frames at once, such that long-term temporally-based objective functions can be employed to correctly learn the motion multi-modality and variances. With our system, long-duration motions can be predicted/synthesized using an open-loop setup where the motion retains the dynamics accurately. It can also be used for denoising corrupted motions and synthesizing new motions with given control signals. We demonstrate that our system can create superior results comparing to existing work in multiple applications.

Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control.

This paper introduces a new generative deep learning network for human motion synthesis and control. Our key idea is to combine recurrent neural networks (RNNs) and adversarial training for human motion modeling. We first describe an efficient method for training an RNN model from prerecorded motion data. We implement RNNs with long short-term memory (LSTM) cells because they are capable of addressing the nonlinear dynamics and long term temporal dependencies present in human motions. Next, we train a refiner network using an adversarial loss, similar to generative adversarial networks (GANs), such that refined motion sequences are indistinguishable from real mocap data using a discriminative network. The resulting model is appealing for motion synthesis and control because it is compact, contact-aware, and can generate an infinite number of naturally looking motions with infinite lengths. Our experiments show that motions generated by our deep learning model are always highly realistic and comparable to high-quality motion capture data. We demonstrate the power and effectiveness of our models by exploring a variety of applications, ranging from random motion synthesis, online/offline motion control, and motion filtering. We show the superiority of our generative model by comparison against baseline models.

Modelling and simulation of sprinters’ health promotion strategy based on sports biomechanics

With the rapid development of the new edge discipline sports biomechanics, modelling and simulation of human motion as one of the cutting-edge research topics in sports biomechanics is receiving more and more attention. Based on the establishment of the three-dimensional simulation system model of human knee joint and the analysis of buckling motion, this paper uses the three-dimensional image registration knee joint motion analysis method to carry out the research on knee flexion movement to solve the knee joint injury problem of sprinters. The results showed that medial collateral ligament injury and meniscus injury were more common in all types of knee joint injuries, accounting for 24% respectively 7% and 22.4%. The incidence of lateral collateral ligament injury was relatively small, accounting for only 5.6%. And the research shows that the foot strength curve recognition method based on biomechanics sprinters knee joint simulation model is an advanced and complete strategy to reduce sprinters knee joint injury, and it can achieve better results in sprinters knee joint nursing.

Experimental Analysis of a Spatialised Audio Interface for People with Visual Impairments

Sound perception is a fundamental skill for many people with severe sight impairments. The research presented in this article is part of an ongoing project with the aim to create a mobile guidance aid to help people with vision impairments find objects within an unknown indoor environment. This system requires an effective non-visual interface and uses bone-conduction headphones to transmit audio instructions to the user. It has been implemented and tested with spatialised audio cues, which convey the direction of a predefined target in 3D space. We present an in-depth evaluation of the audio interface with several experiments that involve a large number of participants, both blindfolded and with actual visual impairments, and analyse the pros and cons of our design choices. In addition to producing results comparable to the state-of-the-art, we found that Fitts’s Law (a predictive model for human movement) provides a suitable metric that can be used to improve and refine the quality of the audio interface in future mobile navigation aids.

Force Shadows: An Online Method to Estimate and Distribute Vertical Ground Reaction Forces from Kinematic Data.

Kinetic models of human motion rely on boundary conditions which are defined by the interaction of the body with its environment. In the simplest case, this interaction is limited to the foot contact with the ground and is given by the so called ground reaction force (GRF). A major challenge in the reconstruction of GRF from kinematic data is the double support phase, referring to the state with multiple ground contacts. In this case, the GRF prediction is not well defined. In this work we present an approach to reconstruct and distribute vertical GRF (vGRF) to each foot separately, using only kinematic data. We propose the biomechanically inspired force shadow method (FSM) to obtain a unique solution for any contact phase, including double support, of an arbitrary motion. We create a kinematic based function, model an anatomical foot shape and mimic the effect of hip muscle activations. We compare our estimations with the measurements of a Zebris pressure plate and obtain correlations of for double support motions and for a walking motion. The presented data is based on inertial human motion capture, showing the applicability for scenarios outside the laboratory. The proposed approach has low computational complexity and allows for online vGRF estimation.

Machine learning model-based two-dimensional matrix computation model for human motion and dance recovery

Many regions of human movement capturing are commonly used. Still, it includes a complicated capturing method, and the obtained information contains missing information invariably due to the human's body or clothing structure. Recovery of motion that aims to recover from degraded observation and the underlying complete sequence of motion is still a difficult task, because the nonlinear structure and the filming property is integrated into the movements. Machine learning model based two-dimensional matrix computation (MM-TDMC) approach demonstrates promising performance in short-term motion recovery problems. However, the theoretical guarantee for the recovery of nonlinear movement information lacks in the two-dimensional matrix computation model developed for linear information. To overcome this drawback, this study proposes MM-TDMC for human motion and dance recovery. The advantages of the machine learning-based Two-dimensional matrix computation model for human motion and dance recovery shows extensive experimental results and comparisons with auto-conditioned recurrent neural network, multimodal corpus, low-rank matrix completion, and kinect sensors methods.

Cartesian Control of Sit-to-Stand Motion Using Head Position Feedback.

Sit-to-stand (STS) motion is an indicator of an individual's physical independence and well-being. Determination of various variables that contribute to the execution and control of STS motion is an active area of research. In this study, we evaluate the clinical hypothesis that besides numerous other factors, the central nervous system (CNS) controls STS motion by tracking a prelearned head position trajectory. Motivated by the evidence for a task-oriented encoding of motion by the CNS, we adopt a robotic approach for the synthesis of STS motion and propose this scheme as a solution to this hypothesis. We propose an analytical biomechanical human CNS modeling framework where the head position trajectory defines the high-level task control variable. The motion control is divided into low-level task generation and motor execution phases. We model CNS as STS controller and its Estimator subsystem plans joint trajectories to perform the low-level task. The motor execution is done through the Cartesian controller subsystem that generates torque commands to the joints. We do extensive motion and force capture experiments on human subjects to validate our analytical modeling scheme. We first scale our biomechanical model to match the anthropometry of the subjects. We do dynamic motion reconstruction through the control of simulated custom human CNS models to follow the captured head position trajectories in real time. We perform kinematic and kinetic analyses and comparison of experimental and simulated motions. For head position trajectories, root mean square (RMS) errors are 0.0118 m in horizontal and 0.0315 m in vertical directions. Errors in angle estimates are 0.55 rad, 0.93 rad, 0.59 rad, and 0.0442 rad for ankle, knee, hip, and head orientation, respectively. RMS error of ground reaction force (GRF) is 50.26 N, and the correlation between ground reaction torque and the support moment is 0.72. Low errors in our results validate (1) the reliability of motion/force capture methods and anthropometric technique for customization of human models and (2) high-level task control framework and human CNS modeling as a solution to the hypothesis. Accurate modeling and detailed understanding of human motion can have significant scope in the fields of rehabilitation, humanoid robotics, and virtual characters' motion planning based on high-level task control schemes.

Character controllers using motion VAEs

A fundamental problem in computer animation is that of realizing purposeful and realistic human movement given a sufficiently-rich set of motion capture clips. We learn data-driven generative models of human movement using autoregressive conditional variational autoencoders, or Motion VAEs. The latent variables of the learned autoencoder define the action space for the movement and thereby govern its evolution over time. Planning or control algorithms can then use this action space to generate desired motions. In particular, we use deep reinforcement learning to learn controllers that achieve goal-directed movements. We demonstrate the effectiveness of the approach on multiple tasks. We further evaluate system-design choices and describe the current limitations of Motion VAEs.

Modeling and Simulation of an Unpowered Lower Extremity Exoskeleton Based on Gait Energy

Aiming at the problem of how to store/release gait energy with high efficiency for the conventional unpowered lower extremity exoskeletons, an unpowered lower-limb exoskeleton is proposed. In the current study, the human motion model is established, and the change rule and recovery/utilization mechanism of gait energy are illustrated. The stiffness and metabolic cost of relevant muscles in lower extremity joints are obtained based on OpenSim software. The results show that stiffness of muscle is increased when muscle concentric contraction generates positive work, but it is reverse when muscle eccentric contraction generates negative work. Besides, metabolic cost of the soleus, gastrocnemius, and tibialis anterior decreased about 31.5%, 34.7%, and 40%, respectively. Metabolic cost of the rectus femoris, tensor fascia lata, and sartorius decreased about 36.3%, 7%, and 5%, respectively, and the total metabolic cost of body decreased about 15.5%, under the exoskeleton conditions. The results of this study can provide a theoretical basis for the optimal design of unpowered lower extremity exoskeleton.

Machine Learning Approaches for Motor Learning: A Short Review

Machine learning approaches have seen considerable applications in human movement modeling, but remain limited for motor learning. Motor learning requires accounting for motor variability, and poses new challenges as the algorithms need to be able to differentiate between new movements and variation of known ones. In this short review, we outline existing machine learning models for motor learning and their adaptation capabilities. We identify and describe three types of adaptation: Parameter adaptation in probabilistic models, Transfer and meta-learning in deep neural networks, and Planning adaptation by reinforcement learning. To conclude, we discuss challenges for applying these models in the domain of motor learning support systems.

Predicting Perturbed Human Arm Movements in a Neuro-Musculoskeletal Model to Investigate the Muscular Force Response.

Human movement is generated by a dynamic interplay between the nervous system, the biomechanical structures, and the environment. To investigate this interaction, we propose a neuro-musculoskeletal model of human goal-directed arm movements. Using this model, we simulated static perturbations of the inertia and damping properties of the arm, as well as dynamic torque perturbations for one-degree-of freedom movements around the elbow joint. The controller consists of a feed-forward motor command and feedback based on muscle fiber length and contraction velocity representing short-latency (25 ms) or long-latency (50 ms) stretch reflexes as the first neuronal responses elicited by an external perturbation. To determine the open-loop control signal, we parameterized the control signal resulting in a piecewise constant stimulation over time for each muscle. Interestingly, such an intermittent open-loop signal results in a smooth movement that is close to experimental observations. So, our model can generate the unperturbed point-to-point movement solely by the feed-forward command. The feedback only contributed to the stimulation in perturbed movements. We found that the relative contribution of this feedback is small compared to the feed-forward control and that the characteristics of the musculoskeletal system create an immediate and beneficial reaction to the investigated perturbations. The novelty of these findings is (1) the reproduction of static as well as dynamic perturbation experiments in one neuro-musculoskeletal model with only one set of basic parameters. This allows to investigate the model's neuro-muscular response to the perturbations that—at least to some degree—represent stereotypical interactions with the environment; (2) the demonstration that in feed-forward driven movements the muscle characteristics generate a mechanical response with zero-time delay which helps to compensate for the perturbations; (3) that this model provides enough biomechanical detail to allow for the prediction of internal forces, including joint loads and muscle-bone contact forces which are relevant in ergonomics and for the development of assistive devices but cannot be observed in experiments.

Effect of rotator cuff muscle activation on glenohumeral kinematics: A cadaveric study

Healthy shoulder function requires the coordination of the rotator cuff muscles to maintain the humeral head’s position in the glenoid. While glenohumeral stability has been studied in various settings, few studies have characterized the effect of dynamic rotator cuff muscle loading on glenohumeral translation during shoulder motion. We hypothesize that dynamic rotator cuff muscle activation decreases joint translation during continuous passive abduction of the humerus in a cadaveric model of scapular plane glenohumeral abduction. The effect of different rotator cuff muscle activity on glenohumeral translation was assessed using a validated shoulder testing system. The Dynamic Load profile is a novel approach, based on musculoskeletal modeling of human subject motion. Passive humeral elevation in the scapular plane was applied via the testing system arm, while the rotator cuff muscles were activated according to the specified force profiles using stepper motors and a proportional control feedback loop. Glenohumeral translation was defined according to the International Society of Biomechanics. The Dynamic load profile minimized superior translation of the humeral head relative to the conventional loading profiles. The total magnitude of translation was not significantly different (0.805) among the loading profiles suggesting that the compressive forces from the rotator cuff primarily alter the direction of humeral head translation, not the magnitude. Rotator cuff muscle loading is an important element of cadaveric shoulder studies that must be considered to accurately simulate glenohumeral motion. A rotator cuff muscle activity profile based on human subject muscle activity reduces superior glenohumeral translation when compared to previous RC loading profiles.

Efficient DropBox Deployment toward Improving Post-Disaster Information Exchange in a Smart City

In the face of a disaster, the already installed gadgets in a smart city can be leveraged to gather post-disaster situational information. However, owing to the typical disruption of cellular and Internet connectivity during disasters, the possibility of transmitting situational information using conventional communication infrastructure is almost ruled out. The networking research community has strongly proposed the use of delay tolerant networks (DTN) in such challenged network scenario. In this article, we exploit the movement of volunteers carrying smartphones in such a scenario to form a DTN and propose a utility-based DropBox deployment scheme toward improving post-disaster situational information exchange. In this scheme, DropBoxes are deployed across the network at high utility locations. Since the effectiveness of the proposed scheme can be evaluated accurately considering an appropriate post-disaster mobility model suitable for smart cities, we present a human movement model for a post-disaster scenario in a smart city. This movement model is shown to have better performance over other competing movement models in a post-disaster smart city environment. An extensive simulation is performed using ONE simulator to evaluate the comparative performance of the proposed DropBox deployment scheme with some state-of-the-art existing deployment schemes using the presented movement model. Simulation results justify the proposed DropBox deployment scheme improves network performances in terms of delivery ratio, overhead ratio, and average residual energy at the cost of tolerable latency.

Methods for Rapid Mobility Estimation to Support Outbreak Response.

When pressed for time, outbreak investigators often use homogeneous mixing models to model infectious diseases in data-poor regions. But recent outbreaks such as the 2014 Ebola outbreak in West Africa have shown the limitations of this approach in an era of increasing urbanization and connectivity. Both outbreak detection and predictive modeling depend on realistic estimates of human and disease mobility, but these data are difficult to acquire in a timely manner. This is especially true when dealing with an emerging outbreak in an under-resourced nation. Weighted travel networks with realistic estimates for population flows are often proprietary, expensive, or nonexistent. Here we propose a method for rapidly generating a mobility model from open-source data. As an example, we use road and river network data, along with population estimates, to construct a realistic model of human movement between health zones in the Democratic Republic of the Congo (DRC). Using these mobility data, we then fit an epidemic model to real-world surveillance data from the recent Ebola outbreak in the Nord Kivu region of the DRC to illustrate a potential use of the generated mobility estimation. In addition to providing a way for rapid risk estimation, this approach brings together novel techniques to merge diverse GIS datasets that can then be used to address issues that pertain to public health and global health security.

SRMM: A Social Relationship-Aware Human Mobility Model

Since human movement patterns are important for validating the performance of wireless networks, several traces of human movements in real life have been collected. However, collecting data about human movements is costly and time-consuming. Moreover, multiple traces are demanded to test various network scenarios. As a result, a lot of synthetic models of human movement have been proposed. Nevertheless, most of the proposed models were often based on random generation, and cannot produce realistic human movements. Although there have been a few models that tried to capture the characteristics of human movement in real life (e.g., flights, inter-contact times, and pause times following the truncated power-law distribution), those models still cannot reflect realistic human movements due to a lack of consideration for social context among people. To address those limitations, in this paper, we propose a novel human mobility model called the social relationship–aware human mobility model (SRMM), which considers social context as well as the characteristics of human movement. SRMM partitions people into social groups by exploiting information from a social graph. Then, the movements of people are determined by considering the distances to places and social relationships. The proposed model is first evaluated by using a synthetic map, and then a real road map is considered. The results of SRMM are compared with a real trace and other synthetic mobility models. The obtained results indicate that SRMM is consistently better at reflecting both human movement characteristics and social relationships.

Construction of three-dimensional human body motion model with muscle-skeletal and cardiovascular system

Construction of three-dimensional human body motion model with muscle-skeletal and cardiovascular system

Research on athlete’s action recognition based on acceleration sensor and deep learning

Due to the difficulty of athletes’ motion recognition, there are few studies on athletes’ specific motion recognition. Based on this, this study uses the acceleration sensor as the carrier, and uses human-computer interaction to transform the action of the athlete into a machine-identifiable action unit. At the same time, this paper combines the actual situation of human body motion to construct a human body motion model and builds a corresponding computer hardware and software platform. Moreover, this paper designs a classification recognition algorithm that can recognize the movement of athletes and builds SVM model based on machine learning for classification and recognition. In addition, in this study, the effectiveness of the algorithm was studied through experimental comparison. Finally, the simulation analysis was carried out to obtain the corresponding research results, and the results were analyzed by combing statistics. The research shows that the proposed algorithm can classify and recognize the collected motion data, and it has certain effects on the theoretical analysis of athletes’ motion recognition. Moreover, the algorithm can perform motion quality analysis and provide theoretical reference for subsequent related research.

Fitness coach: design and implementation of a smart mirror based on automatic image recognition and action model comparison

The smart mirror designed and developed in this study aims to help people who cannot receive professional guidance in completing the standardisation and correction of a series of actions, such as bodybuilding, yoga, and dance. Techniques such as the establishment and recognition of the kinect human motion model, 3D stereoscopic imaging, postural correction based on OpenCV, and the use of voice prompts have been applied to create this smart mirror. The final product is a physical smart mirror that displays corrective images, evaluates the user's actions, and provides helpful suggestions through voice prompts when a user performs actions while facing the mirror.

Research on Intention Recognition Method Based on Radial Basis Function Neural Network

A robot should be endowed with certain collaboration experience to recognize human’s behavioral intention. This paper provides a method based on machine learning to recognize the collaborator’s intention. A radial basis function neural network model was built for offline practice of a robot to recognize intention. Some collaboration skills can be obtained by the robot by building a map between the collaborator’s intention and the system state, deducing human’s intention based on the dynamic characteristics of collaborator and robot and taking the collaborator’s intention as the feedforward information for controlling the robot so as to estimate the human’s intention online based on collaborator’s force and robot’s motion characteristics during collaboration. The proposed method can overcome the difficulties in building the human-robot collaboration model by traditional method, especially the complicated human motion model, and difficulties in estimation of impedance parameters of human body. An experiment was conducted on a motion platform with single degree of freedom. The results prove that the collaborator’s force is reduced while synchronization of human-robot collaboration is improved, so that the compliance of collaborated motion is also improved.