Trajectory Prediction Method Research Articles

Multiagent Multimodal Trajectory Prediction in Urban Traffic Scenarios Using a Neural Network-Based Solution

Trajectory prediction in urban scenarios is critical for high-level automated driving systems. However, this task is associated with many challenges. On the one hand, a scene typically includes different traffic participants, such as vehicles, buses, pedestrians, and cyclists, which may behave differently. On the other hand, an agent may have multiple plausible future trajectories based on complex interactions with the other agents. To address these challenges, we propose a multiagent, multimodal trajectory prediction method based on neural networks, which encodes past motion information, group context, and road context to estimate future trajectories by learning from the interactions of the agents. At inference time, multiple realistic future trajectories are predicted. Our solution is based on an encoder–decoder architecture that can handle a variable number of traffic participants. It uses vectors of agent features as inputs rather than images, and it is designed to run on a physical autonomous car, addressing the real-time operation requirements. We evaluate the method using the inD dataset for each type of traffic participant and provide information about its integration into an actual self-driving car.

An automated driving systems data acquisition and analytics platform

In this paper, an automated driving system (ADS) data acquisition and analytics platform for vehicle trajectory extraction, reconstruction, and evaluation based on connected automated vehicle (CAV) cooperative perception are presented. This platform presents a holistic pipeline from the raw advanced sensory data collection to data processing, which is capable of processing the sensor data from multi-CAVs and extracting the objects’ Identity (ID) number, position, speed, and orientation information in the map and Frenet coordinates. First, the ADS data acquisition and analytics platform are presented. Specifically, the experimental CAVs platform and sensor configuration are shown, and the processing software, including a deep-learning-based object detection algorithm using LiDAR information, a late fusion scheme to leverage cooperative perception to fuse the detected objects from multi-CAVs, and a multi-object tracking method is introduced. To further enhance the object detection and tracking results, high-definition maps consisting of point cloud and vector maps are generated and forwarded to a world model to filter out the objects off the road and extract the objects’ coordinates in Frenet coordinates and the lane information. In addition, to refine trajectories from the object tracking algorithms, a post-processing method is proposed. Given the objects’ information from the object detection and tracking and the world model, a Kalman filter and Chi-square test method are applied to reduce the noise and remove the outlier in the trajectories. Aiming at tackling the ID switch issue of the object tracking algorithm, a fuzzy-logic-based approach is proposed to detect the discontinuous trajectories belonging to the same object. Then, a vehicle-kinematics-based trajectory prediction method is used, and a forward–backward-smoothing technique is applied to reconstruct the trajectory between the discontinuous trajectories. Finally, results, including object detection and tracking and a late fusion scheme, are presented, and the improvements by the post-processing algorithm in terms of noise level and outlier removal are discussed, which confirm the functionality and effectiveness of the proposed holistic data collection and processing platform. In another aspect, the extracted objects’ information and generated HD maps can be used for several purposes in the transportation research community and ADS development community: analyzing the interaction between human-driven vehicles and ADS-equipped vehicles, car-following behavior analysis of ADS-equipped vehicles, traffic flow status analysis and modeling, and scenario generation for ADS testing.

Two-Dimensional Space Turntable Pitch Axis Trajectory Prediction Method Based on Sun Vector and CNN-LSTM Model

A two-dimensional space turntable system has been used to ensure that the Solar X-ray and Extreme Ultraviolet Imager (X-EUVI) can track the Sun stably, and the prediction of the two-dimensional turntable trajectory is an important part of payload health management. Different from the dynamic model using traditional trajectory prediction, we propose a new method for predicting the pitch axis trajectory of the turntable based on the sun vector and a deep learning CNN-LSTM model. First, the ideal solar position of the pitch axis was calculated using the sun vector. Then, the ideal solar position was combined with the running turntable pitch axis motor speed, current, and solar position error signal as the CNN-LSTM model input data. The model parameters were trained and adjusted through test data simulation using Fengyun-3E satellite orbit data. Finally, the next position of the pitch axis was predicted. The test results showed that in the sun vector and CNN-LSTM model, the RMSE value was 0.623 and the MSE value was 0.388. It was better than the LSTM model or CNN model alone and could accurately predict the pitch axis position.

Model identification of ship turning maneuver and extreme short-term trajectory prediction under the influence of sea currents

To improve the accuracy of extreme short-term prediction of ship motion under environmental disturbances, a method to identify the motion model of ship turning maneuver and predict the ship extreme short-term trajectory with the influence of ocean currents is studied. Under the influence of ocean currents, a ship motion response model is identified based on the least squares method, and a ship extreme short-term trajectory prediction model is established. The velocity distribution extraction algorithm, center point restoration algorithm, and weighted derivation algorithm are proposed to extract the current direction and the velocity of the turning test. By integrating the current information and ship response characteristics in the extreme short-term ship prediction model, the extreme short-term segmented prediction of the ship trajectory is realized, and the current information is corrected online according to the prediction errors. The real test results show that, compared with the prediction method based on the ship dynamic model, inertia derivation and current compensation, the proposed extreme short-term trajectory prediction method has higher accuracy. Specifically, the prediction error of the proposed method is reduced by 80.6%, 89.3%, and 54.7% than abovementioned methods, respectively.

An adaptive multivariate Student's t‐process recursive method for hypersonic glide vehicle trajectory prediction

AbstractThe trajectory prediction of the hypersonic glide vehicle (HGV) can provide hit point information for early warning systems, which is of great significance for near‐space defence operations. However, the heavy‐tailed noise caused by abnormal environmental disturbance and abrupt changes in vehicle trajectory seriously affects the accuracy of HGV trajectory prediction. To solve the problem of trajectory prediction for HGV under heavy‐tailed noise, we propose an adaptive multivariate Student's t‐process regression method called aEM‐MVTPR. Firstly, a heavy‐tailed noise model is developed using the Student's t‐distribution. Secondly, a multivariate Student's t‐process regression (MVTPR) for HGV trajectory prediction is derived, and the method can learn the HGV trajectory time series features and mine the correlations among the trajectory variables. Finally, to further improve the method's robustness, we use the accelerated expectation maximisation algorithm and Pearson correlation analysis to adaptively estimate and adjust the initial values of the MVTPR parameters. The simulation experimental results show that the proposed method has more accurate predicted values of trajectory variables than multivariate Gaussian process regression (MVGPR) under heavy‐tailed noise conditions. In addition, the ability of the aEM‐MVTPR to adaptively adjust the parameter makes it more robust than the MVTPR under different noise environments.

METO-S2S: A S2S based vessel trajectory prediction method with Multiple-semantic Encoder and Type-Oriented Decoder

Vessel trajectory prediction plays a vital role in maintaining a safe and effective status in maritime transportation. The development of deep learning provides appropriate mechanisms to predict vessel trajectory using Automatic Identification System (AIS) data. However, besides limited historical position information, much information can be used to enhance deep models, including speed, course, departure time, sailing distance and vessel type. Realistically speaking, these are related to the ships’ endurance, oil storage, tide and work–rest habits. In order to address these problems, we propose a novel vessel trajectory prediction model METO-S2S based on a sequence-to-sequence structure, consisting of the Multiple-semantic Encoder and the Type-Oriented Decoder. After comprehensive data preprocessing and feature engineering, the Multiple-semantic Encoder embeds the sequential inputs of the voyage information into higher-dimensional latent vectors. In addition, the combined vector of vessel type and departure time of a particular ship is fed into the Type-Oriented Decoder treated as guidance information. To verify the efficiency and efficacy of METO-S2S, we employ an AIS dataset in US coastal waters which is more suitable for deep learning models, and implement comparative experiments with several baseline models. The experimental results show that METO-S2S is superior to them both quantitatively and qualitatively.11The code and data of our experiments is available on GitHub https://github.com/AIR-SkyForecast/METO-S2S.

PESO: A Seq2Seq-Based Vessel Trajectory Prediction Method with Parallel Encoders and Ship-Oriented Decoder

Vessel trajectory prediction supports navigation services and collision detection. To maintain safety and efficiency in maritime transportation, vessel trajectory prediction is always an important topic. By using automatic identification system (AIS) data and deep learning methods, the task of vessel trajectory prediction has made significant progress. However, this task is still full of challenges due to the complexity of historical information dependencies and the strong influence of spatial correlations. In this paper, we introduce a novel deep learning model, PESO, based on the structure of Seq2Seq, consisting of Parallel Encoders and a Ship-Oriented Decoder. The Parallel Encoders, including the Location Encoder and the Sailing Status Encoder are designed to integrate more information into feature representation. The Ship-Oriented Decoder is targeted to utilize the Semantic Location Vector (SLV) to guide the prediction, which better represents the spatial correlation of historical track points. In order to verify the efficiency and efficacy of PESO, we conducted comparative experiments with several baseline models. The experimental results demonstrate that PESO is superior to them both quantitatively and qualitatively.

Aircraft Trajectory Prediction for Terminal Airspace Employing Social Spatiotemporal Graph Convolutional Network

Four-dimensional (4-D) trajectory prediction is the core of air traffic management technologies such as flow management, conflict detection and resolution, arrival and departure sequencing, and aircraft abnormal behavior monitoring. The airport terminal airspace has a complex airspace structure, and the flight status of aircraft is changeable, which poses a challenge to trajectory prediction. To this end, aiming at the problem of track prediction in airport terminal area, we propose a 4-D trajectory prediction model of social spatiotemporal graph convolutional neural network (S-STGCNN) based on pattern matching. For each type of flight pattern, an S-STGCNN is trained to improve the robustness. Taking each aircraft as a node of a graph, the spatiotemporal graph convolution is used to extract features of the graph so as to simultaneously characterize the time dependence of trajectories and the interaction between aircraft. The time extrapolation convolutional neural network is used to generate the predicted trajectory. Experimental results show that the trajectory prediction method proposed in this paper has a higher prediction accuracy and a generalization ability than other models.

A trajectory prediction method based on bayonet importance encoding and bidirectional LSTM

Predicting vehicle trajectories can not only provide users with efficient travel suggestions, but also help traffic managers make better decisions. This paper aims to address the next position prediction problem to enable the prediction of the movement trajectory of an individual vehicle by considering where it will go. With this aim, it investigates using deep learning methods for predicting a vehicle’s next position based on its journey history. A deep learning model based on bidirectional long short-term memory (Bi-LSTM) with self-attention mechanism that fuses bayonet importance encoding (Bi-LSTMAI) is proposed, which can well capture both local and global semantic dependency. In specific, word embedding in natural language processing (NLP) is introduced to measure the relevance of bayonets and the PageRank algorithm is improved to obtain the bayonet importance encoding. By using real bayonet data, this research compares the prediction performance of the proposed model with several baselines, including LSTM, GRU, Bi-LSTM and Bi-GRU. The result shows that the proposed model yields better performance in all experimental metrics for the task of trajectory prediction.

A TDV attention-based BiGRU network for AIS-based vessel trajectory prediction

Automatic identification system (AIS) is a vessel-based system for the automatic broadcast and reception of vessel information, and it also supports data for trajectory prediction. Since the vessel's sailing route is flexible and changeable and the AIS broadcast is unconfirmed, the trajectory varies greatly and the original AIS data contains some noisy trajectory, which leads to low prediction accuracy and stability. Therefore, to solve the above problem, this paper proposes a trajectory prediction method based on bidirectional gate recurrent unit (BiGRU) and trajectory direction vector (TDV) with attention mechanism. This paper firstly proposes a TDV to associate latitude and longitude with the course and speed. Then the paper proposes an attention mechanism to self-adaptively update weight to the TDV in different stages to eliminate unreasonable predicted trajectory points. Finally, this paper combines the TDV attention mechanism and the BiGRU network to train a vessel trajectory prediction model.

RDGCN: Reasonably dense graph convolution network for pedestrian trajectory prediction

The pedestrian trajectory prediction remains challenging due to its uncertainty and interference from surrounding pedestrians. There are two deficiencies in previous pedestrian trajectory prediction methods: 1. The temporal correlation of social interaction and the movement factors of groups are ignored; 2. Unreasonable interaction weight allocation. In order to eliminate these two deficiencies, a reasonably dense graph convolution network (RDGCN) was developed in this study. Spatial and temporal graphs were first constructed to model social interactions and movement factors. Then, asymmetric three-dimensional (3D) convolution was employed for the fusion of spatial-temporal information to capture the temporal correlation of social interactions and the movement factors of groups. The RSigmoid function was designed to assign interaction weights and to make the setting of interaction weight more reasonable. Finally, a U-TCN module was designed to estimate two-dimensional Gaussian distribution parameters of the future trajectories. On the ETH and UCY datasets, the proposed method outperformed versus other models in terms of average displacement error and final displacement error, and it was capable of predicting complex social behaviors and movement factors.

Knowledge-Driven Accurate Opponent Trajectory Prediction for Gun-Dominated Autonomous Air Combat

The autonomous air combat (AAC) technique has been a lasting topic for decades. Accurate opponent trajectory prediction provides a fundamental basis for decision making in AAC. In this work, we propose a knowledge-driven scheme for the opponent trajectory prediction problem in one-versus-one gun-dominated within-visual-range (WVR) air combat. A dedicated air combat engagement database is first constructed via skilled human pilots flying WVR air combat. Two baseline algorithms following rule-based and learning-based paradigms are developed and optimized. The knowledge-driven scheme begins with defining handcrafted features extracted from the opponent history movements, and principal components analysis is adopted to compress/refine the features. A generalized regression neural network is then developed to compensate for the residual of the rule-based trajectory prediction method, wherein the refined features are used as inputs, and the residual compensation are used as outputs. Via extensive simulation tests, the proposed scheme shows a more accurate performance as compared with the two other baseline algorithms. To demonstrate the applicability of the proposed scheme, an automatic gun-firing strategy for commencing gun attack in AAC is also illustrated, which justifies the proposed scheme.

Real-Time Trajectory Prediction Method for Intelligent Connected Vehicles in Urban Intersection Scenarios

Intelligent connected vehicles (ICVs) have played an important role in improving the intelligence degree of transportation systems, and improving the trajectory prediction capability of ICVs is beneficial for traffic efficiency and safety. In this paper, a real-time trajectory prediction method based on vehicle-to-everything (V2X) communication is proposed for ICVs to improve the accuracy of their trajectory prediction. Firstly, this paper applies a Gaussian mixture probability hypothesis density (GM-PHD) model to construct the multidimension dataset of ICV states. Secondly, this paper adopts vehicular microscopic data with more dimensions, which is output by GM-PHD as the input of LSTM to ensure the consistency of the prediction results. Then, the signal light factor and Q-Learning algorithm were applied to improve the LSTM model, adding features in the spatial dimension to complement the temporal features used in the LSTM. When compared with the previous models, more consideration was given to the dynamic spatial environment. Finally, an intersection at Fushi Road in Shijingshan District, Beijing, was selected as the field test scenario. The final experimental results show that the GM-PHD model achieved an average error of 0.1181 m, which is a 44.05% reduction compared to the LiDAR-based model. Meanwhile, the error of the proposed model can reach 0.501 m. When compared to the social LSTM model, the prediction error was reduced by 29.43% under the average displacement error (ADE) metric. The proposed method can provide data support and an effective theoretical basis for decision systems to improve traffic safety.

A Random Forest-Based Method for Predicting Borehole Trajectories

Drilling trajectory control technology for near-horizontal directional drilling in coal mines is mainly determined empirically by manual skew data, and the empirical results are only qualitative and variable, meanwhile possessing great instability and uncertainty. In order to improve the accuracy and efficiency of drilling trajectory prediction, this paper investigates a random forest regression-based drilling trajectory prediction method after applying numerous machine learning methods to experimental data for comparison. In the selection of prediction features, this paper replaces all feature variables of the borehole trajectory with feature values with higher cumulative influence weights, and screens out three variables with high importance, azimuth, inclination and bend at the present moment of the drill, as the input variables of the model, and the increments in the borehole in a horizontal direction, left and right direction, and up and down direction at the present moment and the next moment as the output variables of the model. In the model training, the model in this paper uses the bootstrap self-service method resampling technique to collect training sample data, constructs a random forest regression model, and takes the mean value of the decision tree output as the result of the borehole trajectory prediction. To further improve the model accuracy, the paper improves the prediction performance of the model by adjusting the parameters of the random forest model such as tree, depth, minimum sample of leaf nodes, minimum sample number of internal node division, etc. The model is also evaluated by using common machine learning evaluation metrics R2 score, RAE, RMSE and MSE. The experimental results show that the average absolute error of the model drops to 1% on the prediction of the horizontal direction and up and down direction; the average absolute error of the model drops to 9% on the prediction of the left and right direction, and this error rate reaches the error requirement in the actual construction process, so the model can effectively improve the prediction accuracy of borehole trajectory while improving the safety of directional construction.

A Cooperation-Aware Lane Change Method for Automated Vehicles

Lane change for automated vehicles (AVs) is an important but challenging task in complex dynamic traffic environments. Due to difficulties in guaranteeing safety as well as a high efficiency, AVs are inclined to choose relatively conservative strategies for lane change. To avoid the conservatism, this paper presents a cooperation-aware lane change method utilizing interactions between vehicles. We first propose an interactive trajectory prediction method to explore possible cooperations between an AV and the others. Further, an evaluation on safety, efficiency and comfort is designed to make a decision on lane change. Thereafter, we propose a motion planning algorithm based on model predictive control (MPC), which incorporates AV’s decision and surrounding vehicles’ interactive behaviors into constraints so as to avoid collisions during lane change. Quantitative testing results show that compared with the methods without an interactive prediction, our method enhances driving efficiencies of the AV and other vehicles by 14.8% and 2.6%, respectively, which indicates that a proper utilization of vehicle interactions can effectively reduce the conservatism of the AV and promote the cooperation between the AV and others.

Trajectory Prediction Neural Network and Model Interpretation Based on Temporal Pattern Attention

High-precision vehicle trajectory prediction can enable autonomous vehicles to provide a safer and more comfortable trajectory planning and control. Unfortunately, current trajectory prediction methods have difficulty extracting hidden driving features across multiple time steps, which is important for long-term prediction. In order to solve this shortcoming, a temporal pattern attention-based trajectory prediction network, named TP2Net, was proposed, and vehicle of interest inception was established to construct an interaction model among vehicles. Experimental results show a 15% improvement in predictive performance over the previous best method under a 5-s prediction horizon. Moreover, in order to explain why temporal pattern attention was adopted and demonstrate its ability to extract hidden features that are intuitive to human beings, a layer interpretation module was included in TP2Net to quantify the mutual information contained between the input and the intermediate layer output tensor. The results of experiments using naturalistic trajectory datasets indicated that temporal pattern attention can extract three important stages in lane changing, showing that temporal pattern attention can effectively extract hidden features and improve prediction accuracy.

A cognition‐inspired trajectory prediction method for vehicles in interactive scenarios

AbstractTrajectory prediction of the ego vehicle is necessary for the cooperation driving of intelligent vehicles and drivers. Methods based on deep learning can fit complex functions, but they usually focus on vehicles' behavioral characteristics. However, vehicles' trajectories are closely related to the cognition results of drivers. Therefore, based on drivers' cognitive characteristics, a network model is designed to predict vehicle trajectories. Specifically, in the perception stage, featured grids are used that are in the driver's view to encode perceptual information; in the decision stage, convolution and graph attention operations are combined to model the driver's interaction with the surrounding traffic elements; in the motion stage, the elements are constrained in one hidden layer by vehicles' actual control inputs and design the corresponding method to obtain probabilistic results. With experiments in two typical scenarios, including intersection and roundabout, the proposed method can obtain reasonable prediction accuracy and generalizability. Meanwhile, abundant experiments are conducted and the results are compared, which reveal some common problems when predicting vehicle trajectories, particularly based on drivers' cognitive characteristics. These lessons learned from this study are summarized which may be useful for newcomers.

Trajectory Forecasting Using Graph Convolutional Neural Networks Based on Prior Awareness and Information Fusion

Predicting the future trajectories of multiple agents is essential for various applications in real life, such as surveillance systems, autonomous driving, and social robots. The trajectory prediction task is influenced by many factors, including individual historical trajectory, interactions between agents, and the fuzzy nature of an agent’s motion. While existing methods have made great progress on the topic of trajectory prediction, a lot of trajectory prediction methods take into account all pedestrians in the scene when simply modeling the influence of nearby pedestrians, and this inevitably brings redundant information. We propose a pedestrian trajectory prediction model based on prior awareness and information fusion. To make the input information more effective, for the different levels of importance of input trajectory information, we design a time information weighting module to weigh the observed trajectory information differently at different moments based on the original observed trajectory information. To reduce the impact of redundant information on trajectory prediction and to improve interaction between pedestrians, we present a spatial interaction module of multi-pedestrians and a topological graph fusion module. In addition, we use a temporal convolutional network module to obtain the temporal interactions between pedestrians. Compared to Social-STGCNN, the experimental results show that the model we propose reduces the average displacement error (ADE) and final displacement error (FDE) by 32% and 38% in the datasets of ETH and UCY, respectively. Moreover, based on this model, we design an autonomous driving obstacle avoidance system that can effectively ensure the safety of road pedestrians.

Vehicle Trajectory Prediction Considering Driver Uncertainty and Vehicle Dynamics Based on Dynamic Bayesian Network

Vehicle trajectory prediction is a crucial but intricate problem for lateral driving assistance systems because of driver uncertainty. This article presents a probabilistic vehicle-trajectory prediction method based on a dynamic Bayesian network (DBN) model integrating the driver’s intention, maneuvering behavior, and vehicle dynamics. By selecting a most-relevant-feature vector using joint mutual information, we design a Gaussian mixture model- hidden Markov model and employ the model as a node in the DBN to identify the driver’s intention. Then, a reference path is generated using the road information. The uncertainties of drivers are captured in steering- and longitudinal-control using a stochastic driver model and a Markov chain, respectively. A vehicle dynamic model ensures that the predicted vehicle trajectory adheres to the vehicle dynamics, which improves the prediction accuracy. A particle filter is used to recursively estimate the vehicle trajectory, including position coordinates and the lateral distance from the vehicle center of gravity to the road edge. We evaluate the proposed DBN trajectory prediction method in both lane-keeping and lane-changing scenarios based on a dataset collected from a real-time dynamic driving simulator. Results show that the proposed method can achieve accurate long-term trajectory prediction.

Lagrange tracking-based long-term drift trajectory prediction method for Autonomous Underwater Vehicle.

Autonomous Underwater Vehicle (AUV) works autonomously in complex marine environments. After a severe accident, an AUV will lose its power and rely on its small buoyancy to ascend at a slow speed. If the reserved buoyancy is insufficient, when reaching the thermocline, the buoyancy will rapidly decrease to zero. Consequently, the AUV will experience prolonged lateral drift within the thermocline. This study focuses on developing a prediction method for the drift trajectory of an AUV after a long-term power loss accident. The aim is to forecast the potential resurfacing location, providing technical support for surface search and salvage operations of the disabled AUV. To the best of our knowledge, currently, there is no mature and effective method for predicting long-term AUV underwater drift trajectories. In response to this issue, based on real AUV catastrophes, this paper studies the prediction of long-term AUV underwater drift trajectories in the cases of power loss. We propose a three-dimensional trajectory prediction method based on the Lagrange tracking approach. This method takes the AUV's longitudinal velocity, the time taken to reach different depths, and ocean current data at various depths into account. The reason for the AUV's failure to ascend to sea surface lies that the remaining buoyancy is too small to overcome the thermocline. As a result, AUV drifts long time within the thermocline. To address this issue, a method for estimating thermocline currents is proposed, which can be used to predict the lateral drift trajectory of the AUV within the thermocline. Simulation is conducted to compare the results obtained by the proposed method and that in a real accident. The results demonstrate that the proposed approach exhibits small directional and positional errors. This validates the effectiveness of the proposed method.