Accurate Motion Prediction Research Articles

Math-data integrated prediction model for ship maneuvering motion

Highly accurate motion prediction is critical to facilitate safe navigation and autonomous control of a ship. In this paper, a math-data integrated prediction (MDIP) model of ship maneuvering motion is newly developed by combining with mathematical and data-driven modules. The math part is identified by employing variable-order hydrodynamic derivatives which are derived from Taylor expansion. Using extended Kalman filter, an optimal-order mathematical prediction model is obtained. The data-driven part dwells on prediction residuals which are approximated by a least square-support vector machine. Furthermore, integration mechanisms are devised to cohere mathematical and data-driven models as a whole, by deploying summation and neural network approximation, respectively. By comparisons, it is verified that not only order selection but also math-data integration can enhance ship motion prediction accuracy, for which various maneuvering tests are analyzed. The results demonstrate that the proposed MDIP model using math-data integration offers much stronger generalization, thereby paving a new path for ship maneuvering motion prediction.

Geometric phase correction: A direct phase correction method to register low contrast noisy TEM images

A major challenge in the emerging field of low-dose electron microscopy lies in the development of drift correction algorithms against beam-induced specimen motion and compatible with highly noisy transmission electron microscopy (TEM) images. We report here a new drift correction method, namely geometric phase correlation (GPC), to correlate the specimen motion in real space by directly measuring the unwrapped geometric phase shift in the spatial frequency domain of the TEM image (especially from the intensive Bragg spots for crystalline materials) with sub-pixel precision. The GPC method outperforms cross-correlation-based methods in both accuracy of specimen motion prediction from highly noisy TEM movies and computational efficiency of drift calculation from abundant image frames, which holds great promise for diverse applications in low-dose TEM imaging of beam-sensitive materials, such as metal-organic frameworks (MOFs) and covalent organic frameworks (COFs).

The application of body symbols in contemporary art from the perspective of Da Vinci’s theory

Abstract Motion recognition methods can distinguish body symbols and play an important role in real-time safety monitoring of pedestrians. In order to solve the problem of lack of data for deep learning-based action recognition models, a human motion synthesis method with hidden state initialization is proposed. Commonly used human motion synthesis methods utilize Recurrent Neural Network (RNN) to automatically generate subsequent motion sequences by taking several frames of motion data as input. In previous work, the initial hidden state of the RNN is usually set to zero or randomly initialized hidden state, resulting in a jump between the end frame of motion input and the first frame of motion synthesis, which affects the quality of the generated motion. To solve this problem, a method for estimating the initial hidden state estimation is proposed, which takes the initial hidden state as the independent variable, uses the objective function of the neural network as the optimization objective, and uses the gradient descent method to optimize the solution to obtain a suitable initial hidden state. The proposed motion model with initial hidden state estimation reduces the prediction error in the first frame by 63.51% and 6.90%, respectively, and the total error in 10 frames by 50.00% and the total errors of 10 frames were reduced by 50.00% and 4.89%, respectively. The experimental results show that the proposed motion model with initial hidden state estimation has better motion synthesis quality and motion prediction accuracy than the method without initial hidden state estimation; the proposed method improves the quality of motion synthesis by accurately estimating the first frame hidden state of the RNN human motion model, which can provide reliable data support for motion recognition models in real-time security monitoring.

Vision-Based Real-Time Motion Estimation of Ground Vehicles from Aerial Imagery

This research presents vision-based maneuvering object motion estimation in case of occlusion. Unequal dimension Interactive Multiple Model (UDIMM) approach is applied to increase the motion prediction accuracy when no measurement is available. Current deep learning-based multi-object tracking algorithms cannot track the objects when an occlusion exists, even for quite a short time. The main reason is the motion model used in tracking algorithms. If the measurement coming from the object detector is not available, the prediction model propagates the Kalman Filter’s motion model. In this period, the object may maneuver or accelerate/decelerate so that the tracker cannot assign the tracklets successfully. Eventually, the ID switch occurs, and object tracking gets lost. Using the UDIMM approach, track loss due to the ID switch is highly reduced compared to the simple constant velocity motion model used in the StrongSort multi-object tracker algorithm. A two-dimensional simulation environment is created to compare the ground truth data with the estimation results. Performance results of different filters are compared.

Mooring line failure diagnosis and motion control of semi-submersible platform based on the predictive model

Real-time prediction of deterministic platform motions in the coming future seconds was essential. The long-short-term memory networks (LSTM) and the fully connected neural networks were employed to establish the neural networks for motion predictions. The predicted results were compared with those calculated by AQWA. The results demonstrated that the predictive model could offer a fast and accurate prediction of the motion response of the semi-submersible platform. The sliding window model was built to predict motion without real input and monitor mooring line failure. The results showed that the predictive model based on the sliding window model can also achieve accurate prediction of platform motion. With the help of the predictive model, a control module written in Python was activated on the basis of the identification of mooring line failure. The control results show that the proportional-integral-derivative (PID) controller was very good at recovering the kinematic performance after mooring line failure and suppressing the continuous failure of the remaining mooring lines.

Simultaneous hand/wrist motion recognition and continuous grasp force estimation based on nonlinear spectral sEMG features for transradial amputees

Simultaneously and accurately estimating both grasp motion classes and force based on surface electromyogram (EMG) of residual arms is essential for precise grasping with a dexterous prosthetic hand. However, accurate grasping-mode prediction and grasping-force estimation often contradict each other. When performing hand grasping motion, different grasping forces will generate different EMG patterns, which would decay the grasp motion prediction accuracy. In this study, we proposed a framework of fusing motion classification and continuous grasp force estimation based on a new set of nonlinear Mel-Frequency Spectrum features of EMG signals. The performance of the proposed method was tested on eight able bodied (AB) subjects and four transradial amputees (TR) who are the end-users of dexterous prostheses. The experimental results showed that the newly proposed features obtained a significant reduction in the average classification error rate compared to other well-known feature sets, achieving improvements of about 5 % to 28 % in the average classification accuracy across all subjects and force levels. Simultaneously, the proposed features obtained the best continuous grasp force estimation accuracies with R2 (coefficient of determination) of 0.96 and 0.83, and root mean square of maximum voluntary contractions of grasp force (std%GFMVC) of 6.2 % and 10.5 % for the AB and TR subjects, respectively, compared to the previous EMG features. This study suggested that the proposed method would be helpful for the stable and fine manipulation of multifunctional prosthetic hand.

Real-time ship motion prediction based on adaptive wavelet transform and dynamic neural network

Ship motion attitude is the time series data with highly nonlinear and non-stationary characteristics significantly affected by environmental factors. To accurately predict the ship motion attitude in real-time, a ship motion attitude prediction model based on the Adaptive Discrete Wavelet Transform Algorithm (ADWT) and the space-time Residual Recurrent Neural Network (RRNN) with a time-varying structure is proposed. The ADWT decomposes the original data into components easier to predict, each element in the optimal component combination is predicted using the RRNN, the structure and parameters of which are adjusted in real-time with the sliding of data window. The model performance tests are conducted based on the simulation data of the ship motion of DTMB5415. The results show that the subsequences decomposed by the ADWT are easier to predict. Compared with other prediction models, the prediction accuracy of the ADWT-RRNN is the highest under all working conditions, its prediction accuracy and stability of it do not fluctuate significantly over a long prediction period. Hence, the more severe the sea states, the more pronounced the performance advantage is over other models. Finally, a reliable and efficient tool is eventually provided for real-time and accurate prediction of ship motion.

Brain motion networks predict head motion during rest- and task-fMRI.

The capacity to stay still during scanning, which is necessary to avoid motion confounds while imaging, varies markedly between people. Here we investigated the effect of head motion on functional connectivity using connectome-based predictive modeling (CPM) and publicly available brain functional magnetic resonance imaging (fMRI) data from 414 individuals with low frame-to-frame motion (Δd < 0.18 mm). Leave-one-out was used for internal cross-validation of head motion prediction in 207 participants, and twofold cross-validation was used in an independent sample (n = 207). Parametric testing, as well as CPM-based permutations for null hypothesis testing, revealed strong linear associations between observed and predicted values of head motion. Motion prediction accuracy was higher for task- than for rest-fMRI, and for absolute head motion (d) than for Δd. Denoising attenuated the predictability of head motion, but stricter framewise displacement threshold (FD = 0.2 mm) for motion censoring did not alter the accuracy of the predictions obtained with lenient censoring (FD = 0.5 mm). For rest-fMRI, prediction accuracy was lower for individuals with low motion (mean Δd < 0.02 mm; n = 200) than for those with moderate motion (Δd < 0.04 mm; n = 414). The cerebellum and default-mode network (DMN) regions that forecasted individual differences in d and Δd during six different tasks- and two rest-fMRI sessions were consistently prone to the deleterious effect of head motion. However, these findings generalized to a novel group of 1,422 individuals but not to simulated datasets without neurobiological contributions, suggesting that cerebellar and DMN connectivity could partially reflect functional signals pertaining to inhibitory motor control during fMRI.

Characteristic of hydrodynamic derivatives for a multi-hull navigation vessel in dynamic positioning mode

The Quadmaran vessel is a maritime multi-hull vessel developed for water quality observation in aquaculture farms. The vessel’s dynamic positioning (DP) performance is essential for the accuracy of the collected data. The maneuvering characteristics in DP mode have been investigated in our previous research using a quasi-static motion prediction model based on static planar motion mechanism (PMM) tests. However, since the Quadmaran vessel has a unique hull arrangement, the motion characteristics need to be determined with greater accuracy. Therefore, in this study, the maneuvering equations are newly established with higher accuracy hydrodynamic derivatives determined through dynamic PMM tests. The higher order hydrodynamic derivatives are obtained from pure sway, pure yaw and pure yaw with a constant angle tests. Model scale experiments are conducted to establish an advanced mathematical model and compared with the quasi-static assumption results. The result shows significant improvements in the motion prediction under DP mode, and a more accurate motion prediction model is successfully established. This study is a new attempt to utilize the dynamic PMM tests method on a vessel with unique shape and motion characteristics. The advanced mathematical model deepens our understanding of the maneuverability of a multi-hull vessel in DP mode.

A three-stage multi-objective heterogeneous integrated model with decomposition-reconstruction mechanism and adaptive segmentation error correction method for ship motion multi-step prediction

Reliable and accurate ship motion prediction is essential for ship navigation at sea and marine operations. Although previous studies have yielded rich results in the field of ship motion prediction, most of them have ignored the importance of the dynamic characteristics of ship motion for constructing forecasting models. Besides, the limitations of the single model and the autocorrelation characteristics of the residual series are also unfavorable factors that hinder the forecasting performance. To fill these gaps, a multi-objective heterogeneous integration model based on decomposition-reconstruction mechanism and adaptive segmentation error correction method is proposed in this paper for ship motion multi-step prediction. Specifically, the proposed model is divided into three stages, which are decomposition-reconstruction mechanism, multi-objective heterogeneous integration model and adaptive segmentation error correction method. The effectiveness of the proposed model is verified using four sets of real ship motion data collected from two sites in the South China Sea. The evaluation results show that the proposed model can effectively improve the prediction performance and outperforms other traditional models and state-of-the-art models in the field of ship motion prediction. Prospectively, the model proposed in this study can be used as an effective aid to ship warning systems and has the potential for practical application in ship marine operations.

Optimizing Traffic Flow in Smart Cities: Soft GRU-Based Recurrent Neural Networks for Enhanced Congestion Prediction Using Deep Learning

Recently, different techniques have been applied to detect, predict, and reduce traffic congestion to improve the quality of transportation system services. Deep learning (DL) is becoming increasingly valuable for solving critiques. DL applications in transportation have been collected in several recently published surveys over the last few years. The existing research has discussed the cloud environment, which does not provide timely traffic forecasts, which is the cause of frequent traffic accidents. Thus, a solid understanding of the difficulties in predicting congestion is required because the transportation system varies widely between non-congested and congested states. This research develops a bi-directional recurrent neural network (BRNN) using Gated Recurrent Units (GRUs) to extract and classify traffic into congested and non-congested. This research uses a bidirectional recurrent neural network to simulate and forecast traffic congestion in smart cities (BRNN). Urban regions worldwide struggle with traffic congestion, and conventional traffic control techniques have failed miserably. This research suggests a data-driven approach employing BRNN for traffic management in smart cities, which uses real-time data from sensors and linked devices to control traffic more efficiently. The primary measures include predicting traffic metrics such as speed, weather, current, and accident probability. Congestion prediction performance has also been improved by extracting more features such as traffic, road, and weather conditions. The proposed model achieved better measures than the existing state-of-the-art methods. This research also explores an overview and analysis of several early initiatives that have shown promising results; moreover, it explores two potential future research approaches to increase the accuracy and efficiency of large-scale motion prediction.

A hierarchical dynamic Bayesian learning network for EMG-based early prediction of voluntary movement intention

Decoding human action intention prior to motion onset with surface electromyograms (sEMG) is an emerging neuroengineering topic with interesting clinical applications such as intelligent control of powered prosthesis/exoskeleton devices. Despite extensive prior works in the related fields, it remains a technical challenge due to considerable variability of complex multi-muscle activation patterns in terms of volatile spatio-temporal characteristics. To address this issue, we first hypothesize that the inherent variability of the idle state immediately preceding the motion initiation needs to be addressed explicitly. We therefore design a hierarchical dynamic Bayesian learning network model that integrates an array of Gaussian mixture model – hidden Markov models (GMM-HMMs), where each GMM-HMM learns the multi-sEMG processes either during the idle state, or during the motion initiation phase of a particular motion task. To test the hypothesis and evaluate the new learning network, we design and build a upper-limb sEMG-joystick motion study system, and collect data from 11 healthy volunteers. The data collection protocol adapted from the psychomotor vigilance task includes repeated and randomized binary hand motion tasks (push or pull) starting from either of two designated idle states: relaxed (with minimal muscle tones), or prepared (with muscle tones). We run a series of cross-validation tests to examine the performance of the method in comparison with the conventional techniques. The results suggest that the idle state recognition favors the dynamic Bayesian model over a static classification model. The results also show a statistically significant improvement in motion prediction accuracy by the proposed method (93.83±6.41%) in comparison with the conventional GMM-HMM method (89.71±8.98%) that does not explicitly account for the idle state. Moreover, we examine the progress of prediction accuracy over the course of motion initiation and identify the important hidden states that warrant future research.

A first-order generalized pseudo-Bayesian method based on moving horizon estimation for surrounding vehicle states estimation in complex environments

Predicting the motion of surrounding vehicles is important for autonomous vehicle trajectory planning. To ensure the accuracy of motion prediction, it is particularly important to obtain information about the state of surrounding vehicles at the current moment. For this purpose, a first-order generalized pseudo-Bayesian method based on moving horizon estimation (GPB1-MHE) is proposed here. First, for vehicles, pedestrians and obstacles in complex environments, we design several different intention-based motion models to describe multiple maneuvers of vehicles during driving. Second, we replace the Kalman filter (KF) in the first-order generalized pseudo-Bayesian (GPB1) method with moving horizon estimation (MHE) to obtain GPB1-MHE. Compared with GPB1, it makes full use of the historical state information of a vehicle to improve the estimation accuracy. Finally, the effectiveness of the estimation method is verified using four operating conditions, overtaking, overtaken, pedestrian approaching, and pedestrian staying away, in a SCANeR studio and Simulink joint simulation environment.

Ultra-wide band radar for prospective respiratory motion correction in the liver

Objective. T1 mapping of the liver is time consuming and can be challenging due to respiratory motion. Here we present a prospective slice tracking approach, which utilizes an external ultra-wide band radar signal and allows for efficient T1 mapping during free-breathing. Approach. The fast radar signal is calibrated to an MR-based motion signal to create a motion model. This motion model provides motion estimates, which are used to carry out slice tracking for any subsequent clinical scan. This approach was evaluated in simulations, phantom experiments and in vivo scans. Main results. Radar-based slice tracking was implemented on an MR system with a total latency of 77 ms. Moving phantom experiments showed accurate motion prediction with an error of 0.12 mm in anterior-posterior and 0.81 mm in head-feet direction. The model error remained stable for up to two hours. In vivo experiments showed visible image improvement with a motion model error three times smaller than with a respiratory bellow. For T1 mapping during free-breathing the proposed approach provided similar results compared to reference T1 mapping during a breathhold. Significance. The proposed radar-based approach achieves accurate slice tracking and enables efficient T1 mapping of the liver during free-breathing. This motion correction approach is independent from scanning parameters and could also be used for applications like MR guided radiotherapy or MR Elastography.

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Human driving decisions are the leading cause of road fatalities. Autonomous driving naturally eliminates such incompetent decisions and thus can improve traffic safety and efficiency. Deep reinforcement learning (DRL) has shown great potential in learning complex tasks. Recently, researchers investigated various DRL-based approaches for autonomous driving. However, exploiting multi-modal fusion to generate perception and motion prediction and then leveraging these predictions to train a latent DRL has not been targeted yet. To that end, we propose enhancing urban autonomous driving using multi-modal fusion with latent DRL. A single LIDAR sensor is used to extract bird's-eye view (BEV), range view (RV), and residual input images. These images are passed into LiCaNext, a real-time multi-modal fusion network, to produce accurate joint perception and motion prediction. Next, predictions are fed with another simple BEV image into the latent DRL to learn a complex end-to-end driving policy ensuring safety, efficiency, and comfort. A sequential latent model is deployed to learn more compact representations from inputs, leading to improved sampling efficiency for reinforcement learning. Our experiments are simulated on CARLA and evaluated against state-of-the-art DRL models. Results manifest that our method learns a better driving policy that outperforms other prevailing models. Further experiments are conducted to reveal the effectiveness of our proposed approach under different environments and varying weather conditions.

Motion prediction for beating heart surgery with GRU

This work aims to * predict * the * 3D * coordinates of the * point * of * interest * (POI) on * the surface of beating heart in dynamic minimally invasive surgery, which can improve the manoeuvrability of cardiac surgical robots and expand their functions. For accurate and robust POI motion prediction, a deep learning technique, the gated recurrent unit (GRU), is employed to learn the spatio-temporal (ST) correlation of the POI and its auxiliary points (APs) from their past trajectories. For reference, two neural network models that exploit only spatial and temporal correlations, respectively, are also investigated. The prediction accuracy and robustness of the above three models are verified based on the motion datasets of phantom and in vivo hearts collected by da Vinci robots. Source codes and motion data are shared on GitHub (https://github.com/oww-file/ST-GRU). The experimental results show that the proposed ST-GRU model is significantly better than the other two reference models, and it outperforms the state-of-the-art deep learning method on the same datasets.

Motion Prediction for Autonomous Vehicle using Deep Learning Architecture and Transfer Learning

In the current scenario, Autonomous Vehicle (AV) technology has become one of the more futuristic approaches in the automotive industry since it aims to enhance driving safety, driving comfort, and its economy, adding to reduced obstacle collision or traffic accident rate. Motion planning contributes a vital part in autonomous driving, serving as a fundamental building block enabling the AV to move further. Though there are several traditional approaches to implement motion planning, yet challenges remain regarding guaranteed performance and safety under all driving circumstances. With the impressive advancement of deep learning technologies, many researchers have tried to develop end-to-end motion planning approaches using deep learning, which generally employ Deep Neural Networks (DNNs) to directly map the raw sensor data (e.g., point clouds and images) to planned trajectories(e.g., yaw velocity and steering angle). However, accurate motion prediction is still critical for autonomous driving where there’s a transfer learning approach which emphasizes on reusing a trained model for various applications. This could possibly improve the accuracy of motion prediction steadily. Considering this in mind, a transfer learning motion prediction approach for autonomous driving employing the DNN is proposed here, also providing visual element representing the predicted path, by streamlining the use of the toolkit available at hand. The output responses are discussed and compared employing the existing prediction model and the proposed model. The experimental results exhibited improved performance of motion prediction of the proposed transfer learning approach proving better accuracy comparitively with the existing Resnet50 model.

A Hybrid BPNN-GARF-SVR Prediction Model Based on EEMD for Ship Motion

Accurate prediction of ship motion is very important for ensuring marine safety, weapon control, and aircraft carrier landing, etc. Ship motion is a complex time-varying nonlinear process which is affected by many factors. Time series analysis method and many machine learning methods such as neural networks, support vector machines regression (SVR) have been widely used in ship motion predictions. However, these single models have certain limitations, so this paper adopts a multi-model prediction method. First, ensemble empirical mode decomposition (EEMD) is used to remove noise in ship motion data. Then the random forest (RF) prediction model optimized by genetic algorithm (GA), back propagation neural network (BPNN) prediction model and SVR prediction model are respectively established, and the final prediction results are obtained by results of three models. And the weights coefficients are determined by the correlation coefficients, reducing the risk of prediction and improving the reliability. The experimental results show that the proposed combined model EEMD-GARF-BPNN-SVR is superior to the single predictive model and more reliable. The mean absolute percentage error (MAPE) of the proposed model is 0.84%, but the results of the single models are greater than 1%.

User Perceive Realism of Machine Learning-based Drone Dynamic Simulator

The drone will be a commonly used technology by a significant portion of society, and simulating a given drone dynamic will be an essential requirement. There are drone dy-namic simulation models to simulate popular commercial drones. In addition, there are many Newtonian and fluid dynamics-based generic drone dynamic models. However, these models consist of many model parameters, and it is impracticable to evaluate the required model parameters to simulate a custom-made drone. A simple method to develop a machine learning-based dynamic drone simulation model to simulate custom- made drones mitigates the issues mentioned above. Specifically, the authors’ research is associated with the development of a machine learning-based drone dynamic model integrated with a virtual reality environment and validation of the user-perceived physical and behavioural realism of the entire solution. A figure of eight manoeuvring patterns was used to collect the data related to drone behaviour and drone pilot inputs. A Neural Network-based approach was employed to develop the machine learning-based drone dynamic model. Validations were done against real-world drone manoeuvres and user tests. Validation results show that the simulations provided by machine learning are accurate at the beginning and it decreases the accuracy with time. However, users also make mistakes/misjudgments while perceiving the real-world or virtual world. Hence, we explored the user perceive motion prediction accuracy of the simulation environment which is associated with the behavioural realism of the simulation environment. User tests show that the entire simulation environment maintains substantial physical realism.

Prediction of respiratory motion based on sequential embedding combined with relational embedding

To propose a deep learning model for modeling and prediction of the integration of respiratory motion in all directions. The respiratory motion signals in different directions were input into the sequential embedding layer composed of LSTM to capture the sequential dependence of the historical motion state and obtain the sequential embedding representation, which enabled relational embedding in all directions through the self-attention mechanism to obtain the relational embedding representation. The sequential embedding representation and the relational embedding representation were concatenated and input into a prediction layer consisting of a fully connected neural network to generate nonlinear prediction components, which were added to the linear prediction components generated by the autoregressive module parallel to the above structure to generate the final prediction. The model was trained using a 'pre-training + fine-tuning' approach. In the validation experiments, 304 respiratory motion trajectories were used for model pre-training, and 7 evaluation samples were used for model testing. The proposed prediction model achieved more accurate prediction results than other methods. For the 7 evaluation samples with different delay time, the proposed prediction model achieved a reduction of absolute deviations in the 3D directions by over 70%. The proposed model is capable of accurate prediction of respiratory motion and can thus help to reduce system delay in precise radiotherapy.