Motion Prediction Method Research Articles

Bidirectional Transformer GAN for Long-term Human Motion Prediction

The mainstream motion prediction methods usually focus on short-term prediction, and their predicted long-term motions often fall into an average pose, i.e., the freezing forecasting problem [ 27 ]. To mitigate this problem, we propose a novel Bidirectional Transformer-based Generative Adversarial Network (BiTGAN) for long-term human motion prediction. The bidirectional setup leads to consistent and smooth generation in both forward and backward directions. Besides, to make full use of the history motions, we split them into two parts. The first part is fed to the Transformer encoder in our BiTGAN while the second part is used as the decoder input. This strategy can alleviate the exposure problem [ 37 ]. Additionally, to better maintain both the local (i.e., frame-level pose) and global (i.e., video-level semantic) similarities between the predicted motion sequence and the real one, the soft dynamic time warping (Soft-DTW) loss is introduced into the generator. Finally, we utilize a dual-discriminator to distinguish the predicted sequence at both frame and sequence levels. Extensive experiments on the public Human3.6M dataset demonstrate that our proposed BiTGAN achieves state-of-the-art performance on long-term (4 s ) human motion prediction, and reduces the average error of all actions by 4%.

A Non-Equal Time Interval Incremental Motion Prediction Method for Maritime Autonomous Surface Ships

Recent technological advancements facilitate the autonomous navigation of maritime surface ships. The accurate data given by a range of various sensors serve as the primary assurance of a voyage's safety. Nevertheless, as sensors have various sample rates, they cannot obtain information at the same time. Fusion decreases the accuracy and reliability of perceptual data if different sensor sample rates are not taken into account. Hence, it is helpful to increase the quality of the fusion information to precisely anticipate the motion status of ships at the sampling time of each sensor. This paper proposes a non-equal time interval incremental prediction method. In this method, the high dimensionality of the estimated state and nonlinearity of the kinematic equation are taken into consideration. First, the cubature Kalman filter is employed to estimate a ship's motion at equal intervals based on the ship's kinematic equation. Next, a ship motion state predictor based on a long short-term memory network structure is created, using the increment and time interval of the historical estimation sequence as the network input and the increment of the motion state at the projected time as the network output. The suggested technique can lessen the effect of the speed difference between the test set and the training set on the prediction accuracy compared with the traditional long short-term memory prediction method. Finally, comparison experiments are carried out to validate the precision and effectiveness of the proposed approach. The experimental results show that the root-mean-square error coefficient of the prediction error is decreased on average by roughly 78% for various modes and speeds when compared with the conventional non-incremental long short-term memory prediction approach. Additionally, the proposed prediction technology and the traditional approach have virtually the same algorithm times, which may fulfill the real engineering requirements.

A Volterra Adaptive Filtering Method for Polar Motion Prediction Based on Chaotic Time Series

In consideration of the complex time-varying characteristics of polar motion (PM), this paper takes PM as chaotic time series. A Volterra adaptive filter is employed for predicting PM based on the state space reconstruction of delay-coordinate embedding of dynamic system. This method first uses the Least Squares (LS) technology to estimate the harmonic models for the linear trend, Annual and Chandler Wobbles (AW and CW) in PM. The selected LS deterministic models are subsequently used to extrapolate the linear trend, AW, and CW, and obtain the LS residues (the difference between the LS model and PM data themselves). Secondly, the phase space and largest Lyapunov exponent of the LS residues are reconstructed, and calculated by means of the C-C and small data-set algorithm, respectively. Further, a Volterra adaptive filter is designed for generating the extrapolations of the LS residues. The extrapolated LS residues are then added to the LS deterministic models in order to obtain the predicted PM values. The EOP C04 time series released by the International Earth Rotation and Reference Systems Service (IERS) are selected as data base to generate the PM predictions up to 60 days in the future. The results of the predictions are analyzed and compared with those obtained by the Earth Orientation Parameters Prediction Comparison Campaign (EOP PCC) and IERS Bulletin A. The results show that the accuracy of the predictions up to 30 days is comparable with that by the most accurate prediction techniques participating in the EOP PCC for PM, but worse than that by those most accurate techniques beyond 30 days in the future. The results also illustrate that the short-term predictions are better than those published by the IERS Bulletin A. However, the errors of the predictions rapidly increase with the prediction days. It is therefore concluded that the proposed method is a potential technology for short-term PM prediction.

A human motion angle prediction method using a neural network and feature-extraction-processed sEMG obtained in real time to utilize mechanical delay of sEMG

Surface electromyogram (sEMG) has been used to estimate human motions. Feature extraction (FE) has been proposed as a process to decrease noise and extract information about muscle activities from measured raw sEMG signals. Electromechanical delay (EMD) is a time delay between a generation of sEMG and subsequent occurrence of human motions, and sEMG can be obtained in advance of motions occurrence by using this time delay. The purpose of this study is to propose a real time human motion prediction method using a neural network model and FE-processed sEMG signals utilizing EMD. Experimental results show that the proposed method has better prediction results among other methods.

Use of Social Interaction and Intention to Improve Motion Prediction Within Automated Vehicle Framework: A Review

Human errors contribute to 94%(±2.2%) of road crashes resulting in fatal/non-fatal causalities, vehicle damages and a predicament in the pathway to safer road systems. Automated Vehicles (AVs) have been a potential attempt in lowering the crash rate by replacing human drivers with an advanced computer-aided decision-making approach. However, AVs are yet to progress in handling the unprecedented situations involving interactions with other road users. This raises a need for a sophisticated and robust methodological framework to predict human driver interaction and intention. It is of prime importance to develop a constructive knowledge on the existing literature for a proficient forward leap in the field. To address this, we aim to conduct a comprehensive review on motion prediction methods in automated driving context with a special emphasis on model-based and data-driven approaches. Over a hundred studies related to the motion prediction for AVs have been extensively reviewed. This study recommends that the field requires more intricate classification of motion prediction methods, as the conventional three-level categorisation scheme should be upgraded to a profound and present-day context. Therefore, we attempt to provide a clear categorisation of existing motion prediction solutions by adopting four principal strategies: 1. Prediction methods, 2. Classes, 3. Algorithms and 4. Datasets. An all-inclusive summary of the reviewed studies with their respective pros and cons are also presented. Furthermore, we summarise the standard evaluation metrics applied for road users’ intention estimation and trajectory prediction tasks. It is found that the recent studies are built upon multi-agent learning systems with interaction among multiple road users in the same road environment. These methods can provide reliable prediction performance in highly interactive situations over long periods of time. However, the limitation could be at the cost of higher computational complexity in comparison to conventional methods, which are simpler to design and computationally effective. It is also observed that the conventional methods can only operate over a narrow prediction horizon and seldom consider the interactions among the road users. This review contributes to knowledge in validation, addresses the discrepancies, to explicate the ambiguities and to streamline current research for a futuristic perspective beneficiary in motion prediction field.

Improved Prediction of Polar Motions by Piecewise Parameterization

Abstract On seanonal timescale, the variation of Earth rotation is mainly regulated by angular momentum exchanges between the solid Earth and the fluidal atmosphere, ocean and hydrosphere. In the 2nd EOP PCC, we developed Dill2019’s method for polar motion prediction, using piecewise autoagressive parameters. The maximum prediction errors within 90 days are 36 and 16 mas for polar motion x and y components, respectively. Compared with Bulletin A, the mean absolute error of polar motion y prediction is improved by 20% in all timescale, and with a maximum improvement of 49% on the 5th day. Whereas, for polar motion x, the performance is slightly better (2% - 8%) within 30 days but worse (−7%~ −19%) within 30~90 days. We found that the prediction accuracy is very sensitive to the quality of the angular momentum data. For example, on average, the prediction of polar motion y is around 2 times better than polar motion x. In addition, we found the accuracy of 30-90 days prediction is dramatically decreased in the year 2020. We suspect that such deterioration might be due to the pandemic of coronavirus COVID-19, which suppressed global airline activities by more than 60%, then result in a lose of air-borne meteorological data, which are important for weather forecast.

A Hybrid Visual–Haptic Framework for Motion Synchronization in Human–Robot Cotransporting: A Human Motion Prediction Method

In this article, we propose a hybrid visual–haptic framework enabling a robot to achieve motion synchronization in human–robot cotransporting. Visual sensing is employed in capturing human motion in real time. To deal with the inherent delays between the human’s initiative motion and the robot’s responsive motion in cotransporting, a human motion prediction method is developed to make the robot follow human motion proactively. Motion synchronization is achieved when the robot accurately tracks the filtered and predicted human motion. Force sensing is utilized to regulate interaction forces to ensure compliance when motion error between the human and robot is generated. A neural network (NN)-based control is proposed to achieve precise trajectory tracking. Comparative experimental results show that our proposed framework is effective in such cotransporting tasks.

Dynamic Model Informed Human Motion Prediction Based on Unscented Kalman Filter

Human motion prediction is the foundation stone of human–robot collaboration in intelligent manufacturing. The nonlinear and stochastic nature of human motion has made it challenging to predict the motion accurately. Many recent deep-learning-based approaches, e.g., convolutional neural networks or recurrent neural networks (RNNs), have been applied to address this challenge. On the other hand, existing works tend to ignore the importance of human dynamics in motion prediction, especially the effect of muscle force on the motion. This article proposes a novel dynamic model informed motion prediction method. It utilizes an unscented Kalman filter (UKF) to predict the state of the future arm dynamic model such that the future motion of the human arm can be obtained. In particular, the arm dynamic model is developed based on Lagrangian mechanics and represented by differential equations. Embracing the future muscle force predicted by RNN into the differential equations, such a dynamic model is capable of explicitly establishing the intrinsic relation between the future muscle force and the corresponding future arm motion. UKF is leveraged to predict the future joint position and velocity of the human arm based on the dynamic model. Experiments on three motion datasets validate that the proposed prediction method, compared with the traditional RNN-based prediction using skeleton vectors, significantly improves the prediction accuracy regarding elbow and wrist positions.

State Estimation and Motion Prediction of Vehicles and Vulnerable Road Users for Cooperative Autonomous Driving: A Survey

The recent progress in autonomous vehicle research and development has led to increasingly widespread testing of fully autonomous vehicles on public roads, where complex traffic scenarios arise. Along with these vehicles, partially autonomous vehicles, manually-driven vehicles, pedestrians, cyclists, and some animals can be present on the road, to which autonomous vehicles must react. This study focuses on a comprehensive survey of the literature on motion prediction and state estimation of vehicles and VRUs, which are essential for path planning and navigation functionalities of an autonomous vehicle. Motion prediction and state estimation methods utilize the vehicle’s own sensory perception capabilities and information obtained through cooperative perception from V2V and V2X connections. This survey summarizes the significant progress that has been made in both categories, discusses the most promising results to date and outlines critical research challenges that need to be overcome to achieve full autonomy, from an ego vehicle’s perspective in mixed traffic environments.

Predicting respiratory motion using a novel patient specific dual deep recurrent neural networks

Real-time tracking of a target volume is a promising solution for reducing the planning margins and both dosimetric and geometric uncertainties in the treatment of thoracic and upper-abdomen cancers. Respiratory motion prediction is an integral part of real-time tracking to compensate for the latency of tracking systems. The purpose of this work was to develop a novel method for accurate respiratory motion prediction using dual deep recurrent neural networks (RNNs). The respiratory motion data of 111 patients were used to train and evaluate the method. For each patient, two models (Network1 and Network2) were trained on 80% of the respiratory wave, and the remaining 20% was used for evaluation. The first network (Network 1) is a ‘coarse resolution’ prediction of future points and second network (Network 2) provides a ‘fine resolution’ prediction to interpolate between the future predictions. The performance of the method was tested using two types of RNN algorithms : Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The accuracy of each model was evaluated using the root mean square error (RMSE) and mean absolute error (MAE). Overall, the RNN model with GRU- function had better accuracy than the RNN model with LSTM-function (RMSE (mm): 0.4 ± 0.2 versus 0.6 ± 0.3; MAE (mm): 0.4 ± 0.2 versus 0.6 ± 0.2). The GRU was able to predict the respiratory motion accurately (<1 mm) up to the latency period of 440 ms, and LSTM’s accuracy was acceptable only up to 240 ms. The proposed method using GRU function can be used for respiratory-motion prediction up to a latency period of 440 ms.

Learning spectral transform for 3D human motion prediction

In existing motion prediction methods that use graph convolution networks, motion sequences are transformed to a spectral domain, and future motions are predicted through graph spectral filtering for the transformed spectral sequences. However, because the conventional spectral transform method uses a predetermined spectral basis, the motion prediction does not work well for aperiodic or complicated motions. To overcome this problem, we propose a method to learn spectral domain transforms from motion sequences in the training dataset. To this end, two methods are attempted: one for learning the frequency of each spectral basis, and another for learning the values of the basis function directly. Through experiments on representative 3D human motion benchmarks, H3.6M and CMU Mocap, we demonstrate that both of the proposed methods consistently outperform the baseline method. In particular, the method of directly learning the basis function outperforms the state-of-the-art methods. We also demonstrate that the proposed method yields realistic predictions, even for aperiodic and complicated action categories.

A Lightweight Long-Term Vehicular Motion Prediction Method Leveraging Spatial Database and Kinematic Trajectory Data

Long-term vehicular motion prediction is a crucial function for both autonomous driving and advanced driver-assistant systems. However, due to the uncertainties of vehicle dynamics and complexities of surroundings, long-term motion prediction is never trivial work. As they combine effects of humans, vehicles and environments, kinematic trajectory data reflect several aspects of vehicles’ spatial behaviors. In this paper, we propose a novel method that leverages spatial database and kinematic trajectory data to achieve long-term vehicular motion prediction in a lightweight way. In our system, a spatial database system is initially embedded in an extended Kalman filter (EKF) framework. The spatial kinematic trajectory data are managed through the database and directly used in motion prediction; namely, weighted means are derived from the spatially retrieved kinematic data and used to update EKF predictions. The proposed method is validated in the real world. The experiments indicate that different weighting methods make a slight accuracy difference. Our method is not data-and-computation-consumed; its performance is acceptable in the limited data conditions and its prediction accuracy is improved as the size of used data sets increases; our method can predict in real time. The efficiency of an unscented Kalman filter (UKF) is compared with that of the EKF. The results show that the UKF can hardly meet real-time requirements.

A new polar motion prediction method combined with the difference between polar motion series

After the first Earth Orientation Parameters Prediction Comparison Campaign (1st EOP PCC), the traditional method using least-squares extrapolation and autoregressive (LS + AR) models was considered as one of the polar motion prediction methods with higher accuracy. The traditional method predicts individual polar motion series separately, which has a single input data and limited improvement in prediction accuracy. To address this problem, this paper proposes a new method for predicting polar motion by combining the difference between polar motion series. The X, Y, and Y-X series were predicted separately using LS + AR models. Then, the new forecast value of X series is obtained by combining the forecast value of Y series with that of Y-X series; the new forecast value of Y series is obtained by combining the forecast value of X series with that of Y-X series. The hindcast experimental comparison results from January 1, 2011 to April 4, 2021 show that the new method achieves a maximum improvement of 12.95% and 14.96% over the traditional method in the X and Y directions, respectively. The new method has obvious advantages compared with the differential method. This study tests the stability and superiority of the new method and provides a new idea for the research of polar motion prediction.

QMEDNet: A quaternion-based multi-order differential encoder–decoder model for 3D human motion prediction

In order to deal with the sequence information in the task of 3D human motion prediction effectively, many previous methods seek to predict the motion state of the next moment using the traditional recurrent neural network in Euclidean space. However human motion representation in Euclidean space has high distortion and shows a weak semantic expression when using deep learning models. In this work, we try to process human motion by mapping Euclidean space into a Hypercomplex vector space. We propose a novel model based on quaternion to predict the three-dimensional motion of a human body. The core idea of this study is to use the fusion information to understand and process the human motion state in quaternion space. The multi-order differential information is fused both in the encoder and decoder of feature extraction and mapped to the quaternion space, respectively. The encoder takes graph convolution as the basic unit and the decoder adopts gated recurrent units. Numerous experiments have been carried out to prove that the multi-order information in quaternion space can help build a more reasonable description for 3D human motion. The performance of the proposed QMEDNet is superior to most of the advanced short and long-term motion prediction methods in both public datasets, Human 3.6M and CMU Mocap.

Deep learning method for 3-DOF motion prediction of unmanned surface vehicles based on real sea maneuverability test

Motion prediction in a real sea area is a relatively difficult problem due to the lack of reliable measurement of the environment and an accurate model of ship motion under external disturbance. A deep learning method is applied in this study for the modeling of 3-DOF motion prediction under a real sea environment with the influence of currents and waves. The network with angle feature and the network with trigonometric feature are validated for the identification of the influence of the environment regarding winds, waves, and currents, in which the selection of activation function is further discussed. The developed method is validated by learning the data generated through the turning test of JARI-USV in a real sea area in Rizhao, Shandong. Analysis on the turning radius and the drift distance of the integrated track shows maximum relative errors of 5.44% and 9.09%, and the derived drift angle has a maximum absolute error of 4.84°. The prediction results verify the ability of the designed network to capture the environment influence on ship motion, as well as the turning feature of the ship itself from the maneuvering test data in a real sea trial.

Predictive Simulation for Surface Fault Occurrence Using High-Performance Computing

Numerical simulations based on continuum mechanics are promising methods for the estimation of surface fault displacements. We developed a parallel finite element method program to perform such simulations and applied the program to reproduce the 2016 Kumamoto earthquake, where surface rupture was observed. We constructed an analysis model of the 5 × 5 × 1 km domain, including primary and secondary faults, and inputted the slip distribution of the primary fault, which was obtained through inversion analysis and the elastic theory of dislocation. The simulated slips on the surface were in good agreement with the observations. We then conducted a predictive simulation by inputting the slip distributions of the primary fault, which were determined using a strong ground motion prediction method for an earthquake with a specified source fault. In this simulation, no surface slip was induced in the sub-faults. A large surface slip area must be established near a sub-fault to induce the occurrence of a slip on the surface.

A comprehensive lateral motion prediction method of surrounding vehicles integrating driver intention prediction and vehicle behavior recognition

In order to solve the accuracy problem of the future motion prediction of the surrounding vehicles with different types of drivers, this paper proposes a comprehensive lateral motion prediction method that combines driver intention prediction and vehicle behavior recognition. For different drivers, different driver optimization models are established: personal optimal model and system optimal model. Then, the driver’s intention prediction probability is obtained through the game theory. Different from the traditional game theory, we apply different model for different driver on the premise of getting the type of driver instead of using the Nash equilibrium for all players. According to the results of driver intention prediction and the vehicle behavior recognition, the optimized polynomial trajectory is used to obtain the driver’s intention prediction trajectory and the vehicle behavior recognition trajectory. Next, the Nash-optimization function of the intention prediction trajectory and the behavior recognition trajectory are established respectively according to the trajectory error, and the balance between the two is our comprehensive trajectory. In order to verify the effectiveness of the proposed method, we conducted simulations under mandatory lane changing and discretionary lane changing conditions. The simulation results show that our algorithm can accurately predict the future motion of the vehicle under different drivers, different speeds and different gaps between vehicles. Its prediction performance is much better than other algorithms.

A Multi-Agent Motion Prediction and Tracking Method Based on Non-Cooperative Equilibrium

A Multi-Agent Motion Prediction and Tracking method based on non-cooperative equilibrium (MPT-NCE) is proposed according to the fact that some multi-agent intelligent evolution methods, like the MADDPG, lack adaptability facing unfamiliar environments, and are unable to achieve multi-agent motion prediction and tracking, although they own advantages in multi-agent intelligence. Featured by a performance discrimination module using the time difference function together with a random mutation module applying predictive learning, the MPT-NCE is capable of improving the prediction and tracking ability of the agents in the intelligent game confrontation. Two groups of multi-agent prediction and tracking experiments are conducted and the results show that compared with the MADDPG method, in the aspect of prediction ability, the MPT-NCE achieves a prediction rate at more than 90%, which is 23.52% higher and increases the whole evolution efficiency by 16.89%; in the aspect of tracking ability, the MPT-NCE promotes the convergent speed by 11.76% while facilitating the target tracking by 25.85%. The proposed MPT-NCE method shows impressive environmental adaptability and prediction and tracking ability.

Deep Learning for 3D Human Motion Prediction: State-of-the-Art and Future Trends

Due to the success of deep learning in wide range of computer vision and computer graphics tasks, there is an increasing number of developed methods leveraging deep neural networks to solve human motion prediction. Recent motion prediction methods focus on solving many issues to predict accurate and natural human motion in temporal domain. In this study, we present a comprehensive survey of deep-learning-based human motion prediction methods. First, we define the human motion prediction problem and the scope of this study. We then provide related background knowledge and a comprehensive list of motion prediction methods based on our proposed classification. Next, we provide a complete survey of the characteristics widely used in the literature and explain the evaluation processes. Finally, we presented a quantitative comparison of recent studies and address the remaining unsolved issues while exploring possible research directions for future research.

A Study on the Number of Preferable Neighborhood Vectors When Using the Improved Jacobi Matrix Estimation Method for Roll Motion Prediction of Small Boat

A Study on the Number of Preferable Neighborhood Vectors When Using the Improved Jacobi Matrix Estimation Method for Roll Motion Prediction of Small Boat