Long-term Trajectory Prediction Research Articles

Graph-driven multi-vessel long-term trajectories prediction for route planning under complex waters

Current collision avoidance algorithms used in autonomous navigation vessels are hindered by limited long-term visibility of dynamic obstacles, resulting in overly cautious and frequent avoidance maneuvers that can actually increase the risk of collisions in complex waterways. Therefore, this paper proposed a novel graph-driven multi-vessel long-term trajectory prediction method (termed GMLTP), which extends the length of real-time and accurate prediction of other vessel trajectories. GMLTP can enhance collision avoidance algorithms by enabling more anticipatory trajectory prediction, ultimately supporting real-time route planning in creating safer, more efficient, and optimized routes. Specifically, GMLTP consists of a multi-graph spatial convolution (MGSC) layer and a probability sparse (ProbSparse) self-attention Transformer. MGSC is a finer spatial interactions extractor, which supports the modeling of spatial dependency evolution on longer sequences and thus improves the perception of long-term trends, while ProbSparse self-attention Transformer is used to reduce the computation of learning long-term global time dependencies and thus extending the predictable length. Extensive experimental results indicate that GMLTP exhibits better accuracy and generalization in prediction tasks beyond 48-timesteps (each time step is 10 s), which can provide better assistance to current real-time route planning tasks.

Vehicle trajectory prediction based on cross-attention and multilevel spatio-temporal features

Accurately predicting vehicle trajectories is crucial for motion planning in autonomous driving. Some existing trajectory prediction models employ Long Short-Term Memory (LSTM) networks to encode trajectory information, but this approach may lead to information loss. Additionally, the sparsity of the grid representation may limit the model’s ability to extract vehicle interaction features. To address these issues, this paper proposes a vehicle trajectory prediction model based on cross-attention and multilevel spatio-temporal features. The proposed model incorporates a spatio-temporal feature extraction module designed to capture the dynamic evolution of vehicle states across both time and space. Furthermore, it employs a temporal cross-attention module that reuses hidden states to select spatio-temporal features closely related to the target vehicle’s historical state, thus compensating for the temporal information loss during the encoding process. Additionally, a spatial cross-attention module, informed by social context, is deployed to analyze the dynamic interactions among vehicles. This module filters spatio-temporal interaction features with a significant impact on the target vehicle’s trajectory. The model is trained and tested using the NGSIM dataset and the highD dataset. Compared with other models, the model presented in this paper shows higher prediction accuracy in long-term trajectory prediction.

An enhanced motion planning approach by integrating driving heterogeneity and long-term trajectory prediction for automated driving systems: A highway merging case study

Navigating automated driving systems (ADSs) through complex driving environments is difficult. Predicting the driving behavior of surrounding human-driven vehicles (HDVs) is a critical component of an ADS. This paper proposes an enhanced motion-planning approach for an ADS in a highway-merging scenario. This method utilizes the results of two aspects: the driving behavior and long-term trajectory of surrounding HDVs, which are coupled using a hierarchical model that is used for the motion planning of an ADS to improve driving safety. An unsupervised clustering algorithm is utilized to classify HDV drivers into two categories: aggressive and normal, as part of predicting their driving behaviors. Subsequently, a logistic regression model is employed for driving style prediction. For trajectory prediction, a transformer-based model that concentrated parallelization computations on longer sequence predictions via a self-attention mechanism is developed. Based on the predicted driving styles and trajectories of the surrounding HDVs, an intelligent decision-making strategy is utilized for ADS motion planning. Finally, real-world traffic data collected from drones on a highway ramp in Xi’an is used as a case study to train and evaluate the proposed model. The results demonstrate that the proposed approach can predict HDVs merging trajectories with a mean squared error (MSE) smaller than 0.125 at 30 s away from the merging point, outperforming existing approaches in terms of predictable duration and accuracy. Furthermore, it exhibited safety improvements by adjusting the ADS motion state in advance with good predictive power for the surrounding HDVs’ motion. For both normal and aggressive driving styles, the ADS trajectories reached optimal safety at a prediction horizon of at least 10 s and over 6.67 s, respectively, when evaluating both time-to-collision (TTC) and deceleration rate to avoid a crash (DRAC) metrics. This indicates that our proposed procedures have improved safety for AVs with long-term predictive capabilities (i.e., over 6 s) for surrounding HDVs. Interestingly, the safety performance does not continue to improve with the increase in prediction horizon once the optimal safety performance (highest TTC and lowest DRAC) has been achieved for both normal and aggressive styles. This suggests that an excessively long prediction horizon (e.g., 30 s) is not always beneficial for safety performance of ADS in our case study. When ADS encounter aggressive HDVs, having a predicted surrounding HDV trajectory can extremely reduce crash risk and maintain a safe situation. This demonstrates that planning ADS trajectories with HDV predictions has significant effects on safety, especially for aggressive driving styles as compared to normal driving styles.

Non-probability sampling network based on anomaly pedestrian trajectory discrimination for pedestrian trajectory prediction

Pedestrian trajectory prediction in first-person view is an important support for achieving fully automated driving in cities. However, existing pedestrian trajectory prediction methods still have significant shortcomings in terms of pedestrian trajectory diversity, dynamic scene constraints, and dependence on long-term trajectory prediction. We proposes a non-probability sampling network based on pedestrian trajectory anomaly recognition (ADsampler) to predict multiple possible future pedestrian trajectories. First, by incorporating pose and optical flow information, ADsampler models the multi-dimensional motion characteristics of pedestrians based on observed trajectory information and discriminates trajectory states. The sampling range in the Gaussian latent space is determined based on the recognition results. Next, velocity and yaw information of the car are introduced to model the car's motion state. A subtraction fusion network is employed to remove redundant image feature constraints in highly dynamic scenes. Finally, ADsampler utilizes a novel trajectory decoding network that combines the position encoding capability of GRU with the long-term dependency capturing ability of Transformer to decode and predict the fused features. we evaluate our model on crowded videos in the public datasets JAAD, PIE, ETH and UCY. Experiments demonstrate that the proposed method outperforms state-of-the-art approaches in prediction accuracy.

VOSTN: Variational One-shot Transformer Network for Pedestrian Trajectory Prediction

The accurate and reliable prediction of pedestrian future trajectories is of crucial significance for ensuring the safe navigation of autonomous driving systems. This paper introduces a novel approach called the Variational One-Shot Transformer Network (VOSTN) for the prediction of future trajectories within the 2D on-board domain. VOSTN presents an innovative method called the One-Shot Generation Block, which aims to generate queries simultaneously in order to predict future trajectories in a parallel manner. The utilization of the parallel prediction effectively addresses the issue of error accumulation resulting from autoregression, improves the efficiency of inference time, and enhances the precision of long-term trajectory prediction. And the cross-attention module investigates the inter-relationship between trajectory and ego-motion. Given the inherent stochasticity of pedestrian movement, we employ the Conditional Variational AutoEncoder to forecast available multimodal trajectories. Experimental results demonstrate that our model effectively exploits the information associated with trajectory and ego-motion, leading to the acquisition of more comprehensive feature representations. Moreover, our model outperforms the performance of existing methods on the two 2D on-board domain datasets. Our deterministic/multimodal prediction models show a reduction in the bounding box center final displacement error by 8% / 9% and 0.7% / 3% on PIE and JAAD, respectively, when compared to the most optimal baseline.

A Novel Multimodal Long-Term Trajectory Prediction Scheme for Heterogeneous User Behavior Patterns

A Novel Multimodal Long-Term Trajectory Prediction Scheme for Heterogeneous User Behavior Patterns

Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling

Extracting the rules of real-world multi-agent behaviors is a current challenge in various scientific and engineering fields. Biological agents independently have limited observation and mechanical constraints; however, most of the conventional data-driven models ignore such assumptions, resulting in lack of biological plausibility and model interpretability for behavioral analyses. Here we propose sequential generative models with partial observation and mechanical constraints in a decentralized manner, which can model agents’ cognition and body dynamics, and predict biologically plausible behaviors. We formulate this as a decentralized multi-agent imitation-learning problem, leveraging binary partial observation and decentralized policy models based on hierarchical variational recurrent neural networks with physical and biomechanical penalties. Using real-world basketball and soccer datasets, we show the effectiveness of our method in terms of the constraint violations, long-term trajectory prediction, and partial observation. Our approach can be used as a multi-agent simulator to generate realistic trajectories using real-world data.

Long-Term Trajectory Prediction of Hypersonic Glide Vehicle Based on Physics-Informed Transformer

Long-Term Trajectory Prediction of Hypersonic Glide Vehicle Based on Physics-Informed Transformer

An Interacting Multiple Model for Trajectory Prediction of Intelligent Vehicles in Typical Road Traffic Scenario.

This article presents an interacting multiple model (IMM) for short-term prediction and long-term trajectory prediction of an intelligent vehicle. This model is based on vehicle's physics model and maneuver recognition model. The long-term trajectory prediction is challenging due to the dynamical nature of the system and large uncertainties. The vehicle physics model is composed of kinematics and dynamics models, which could guarantee the accuracy of short-term prediction. The maneuver recognition model is realized by means of hidden Markov model, which could guarantee the accuracy of long-term prediction, and an IMM is adopted to guarantee the accuracy of both short-term prediction and long-term prediction. The experiment results of a real vehicle are presented to show the effectiveness of the prediction method.

Long-Term Trajectory Prediction for Oil Tankers via Grid-Based Clustering

Vessel trajectory prediction is an important step in route planning, which could help improve the efficiency of maritime transportation. In this article, a high-accuracy long-term trajectory prediction algorithm is proposed for oil tankers. The proposed algorithm extracts a set of waymark points that are representative of the key traveling patterns in an area of interest by applying DBSCAN clustering to historical AIS data. A novel path-finding algorithm is then developed to sequentially identify a subset of waymark points, from which the predicted trajectory to a fixed destination is produced. The proposed algorithm is tested using real data offered by the Danish Maritime Authority. Numerical results demonstrate that the proposed algorithm outperforms state-of-the-art vessel trajectory prediction algorithms and is able to make high-accuracy long-term trajectory predictions.

Predictive hierarchical reinforcement learning for path-efficient mapless navigation with moving target

Deep reinforcement learning (DRL) has been proven as a powerful approach for robot navigation over the past few years. DRL-based navigation does not require the pre-construction of a map, instead, high-performance navigation skills can be learned from trial-and-error experiences. However, recent DRL-based approaches mostly focus on a fixed navigation target. It is noted that when navigating to a moving target without maps, the performance of the standard RL structure drops dramatically on both the success rate and path efficiency. To address the mapless navigation problem with moving target, the predictive hierarchical DRL (pH-DRL) framework is proposed by integrating the long-term trajectory prediction to provide a cost-effective solution. In the proposed framework, the lower-level policy of the RL agent learns robot control actions to a specified goal, and the higher-level policy learns to make long-range planning of shorter navigation routes by sufficiently exploiting the predicted trajectories. By means of making decisions over two level of policies, the pH-DRL framework is robust to the unavoidable errors in long-term predictions. With the application of deep deterministic policy gradient (DDPG) for policy optimization, the pH-DDPG algorithm is developed based on the pH-DRL structure. Finally, through comparative experiments on the Gazebo simulator with several variants of the DDPG algorithm, the results demonstrate that the pH-DDPG outperforms other algorithms and achieves a high success rate and efficiency even though the target moves fast and randomly.

Interaction-Aware Personalized Trajectory Prediction for Traffic Participant Based on Interactive Multiple Model

Trajectory prediction for traffic participants is a critical task for autonomous vehicles. The long-term trajectory prediction is challenging due to limited data and the dynamic characteristics of traffic participants. This paper presents an innovative interactive multiple model algorithm considering inter-vehicle interaction and driving behavior for the traffic participant's short-term and long-term trajectory prediction. The field experiment is conducted to acquire the human driver data, which is then preprocessed and analyzed with statistical methods. The clustering result of the critical gap is used to include the interactions between them, on which the gap satisfaction probability function is designed and aimed at describing the satisfaction probability of the current lane. The driving behavior is another promising candidate to improve the long-term prediction accuracy. The clustering results of the lane change duration are used to establish the lane changing models considering the driving behavior, the driving behavior probability function is designed based on the probability of each model. Then the two functions are incorporated into the adaptive transition probability matrix, where the quantitative probabilistic relations between the gap satisfaction probability and the driving behavior probability are established. The adaptive transition probability matrix is then used in the interactive multiple model algorithm. Based on the improved interactive multiple model, the personalized trajectory prediction for the traffic participant is obtained. The effectiveness of the framework is validated by simulation and field experiment.

Vehicle Trajectory Prediction Considering Driver Uncertainty and Vehicle Dynamics Based on Dynamic Bayesian Network

Vehicle trajectory prediction is a crucial but intricate problem for lateral driving assistance systems because of driver uncertainty. This article presents a probabilistic vehicle-trajectory prediction method based on a dynamic Bayesian network (DBN) model integrating the driver’s intention, maneuvering behavior, and vehicle dynamics. By selecting a most-relevant-feature vector using joint mutual information, we design a Gaussian mixture model- hidden Markov model and employ the model as a node in the DBN to identify the driver’s intention. Then, a reference path is generated using the road information. The uncertainties of drivers are captured in steering- and longitudinal-control using a stochastic driver model and a Markov chain, respectively. A vehicle dynamic model ensures that the predicted vehicle trajectory adheres to the vehicle dynamics, which improves the prediction accuracy. A particle filter is used to recursively estimate the vehicle trajectory, including position coordinates and the lateral distance from the vehicle center of gravity to the road edge. We evaluate the proposed DBN trajectory prediction method in both lane-keeping and lane-changing scenarios based on a dataset collected from a real-time dynamic driving simulator. Results show that the proposed method can achieve accurate long-term trajectory prediction.

Long-Term Trajectory Prediction Model Based on Transformer

Long-Term Trajectory Prediction Model Based on Transformer

A Probabilistic Architecture of Long-Term Vehicle Trajectory Prediction for Autonomous Driving

In mixed and dynamic traffic environments, accurate long-term trajectory forecasting of surrounding vehicles is one of the indispensable preconditions for autonomous vehicles (AVs) to accomplish reasonable behavioral decisions and guarantee driving safety. In this paper, we propose an integrated probabilistic architecture for long-term vehicle trajectory prediction, which consists of a driving inference model (DIM) and a trajectory prediction model (TPM). The DIM is designed and employed to accurately infer the potential driving intention based on a dynamic Bayesian network. The proposed DIM incorporates the basic traffic rules and multivariate vehicle motion information. To further improve the prediction accuracy and realize uncertainty estimation, we develop a Gaussian process (GP)-based TPM, considering both the short-term prediction results of the vehicle model and the driving motion characteristics. Afterward, the effectiveness of our novel approach is demonstrated by conducting experiments on a public naturalistic driving dataset under lane-changing scenarios. The superior performance on the task of long-term trajectory prediction is presented and verified by comparing with other advanced methods.

Structural Transformer Improves Speed-Accuracy Trade-Off in Interactive Trajectory Prediction of Multiple Surrounding Vehicles

Fast and accurate long-term trajectory prediction of surrounding vehicles (SVs) is critical to autonomous driving systems. In high-density traffic flows, strongly correlated vehicle behaviors require considering the interactions among multiple SVs when predicting their future trajectories. However, existing interactive prediction methods, most based on Long Short-Term Memory (LSTM), are suffering from slow prediction because they analyze SVs one by one and analyze trajectory sequence node by node. This paper presents a fast interactive trajectory prediction method called Structural Transformer which learns both spatial and temporal dependencies among multiple SVs in parallel. Specifically, our model first removes the internal states and loops of LSTM and replaces with a weighted self-reference mapping to realize parallel computation. Then, it embeds the relative spatial information of multiple SVs into trajectory states and reorganizes the self-reference mapping with neighbor-only interaction masks to achieve interactive prediction. Results on the NGSIM dataset show satisfyingly speed and accuracy performance on long-term trajectory prediction of multiple SVs. The longitudinal and lateral errors are reduced to 2.67m and 0.25m over 5s time horizon. The computational time of each step is only 12ms on a 2080ti GPU, which is over 4 times faster than the Structural LSTM.

Awareness on Present and Future Trajectory of Vehicle Using Multiple Hypotheses in the Mixed Traffic of Intersection

In the transition period, autonomous vehicles are mixed with unconnected traffic occupants, such as non-autonomous vehicles and pedestrians, resulting in a major hurdle toward autonomy in urban areas, especially at intersections. In this context, the cooperative-intelligent transportation system (C-ITS) affords a promising solution to achieve a breakthrough with its omniscient sensors network and computing capability. From the perspective of a C-ITS-based service, the trajectory of non-autonomous vehicle is a critical uncertainty that resides at the intersection. Therefore, this paper proposes a unique interactive framework, which is installed in the edge server of C-ITS and can estimate the present trajectories and predict the future trajectories of the non-autonomous vehicles at intersections. The proposed framework was based on multiple hypotheses of possible maneuvers that formed the confined prior set to reduce the high uncertainties posed by the complicated environment of the urban intersection. The resulting all-in-one framework provided a stable long-term trajectory prediction with intrinsic maneuver classification and improved tracking in an integrated way by incorporating the interactions between the multiple hypotheses. This situation awareness can assist autonomous vehicles to drive safely and defensively. The proposed framework was verified using a dataset collected at a real urban intersection.

Game Algorithm of Intelligent Driving Vehicle Based on Left-Turn Scene of Crossroad Traffic Flow.

Vehicle networking and autonomous driving are hot areas of scientific research today, and they complement each other and play an important role in people's intelligent travel. Intelligent driving vehicle can enhance road safety, effectively reduce traffic flow and fuel consumption, and promote the overall social development. It has great application value in urban traffic system. The traffic condition of a city directly affects the economic development of the city and the improvement of people's quality of life. As the “core” of the urban traffic network, intersections are the frequent places where traffic jams occur. Game theory, as a win-win theory, mainly solves the problem of multiperson and multi-objective with contradictory objective functions and can be used to study the optimal signal control strategy. Aiming at this problem, the potential conflict behaviors of intelligent driving vehicles when turning left at urban intersections are analyzed and a decision model is established. A long-term trajectory prediction model of straight vehicles is established based on the Gaussian process regression model (GPR) considering the vehicle motion pattern. Combined with trajectory prediction, a decision-making process (model) for intelligent driving vehicles based on conflict resolution and a multifactor driving action selection method are proposed. A coordination algorithm based on game theory is designed for conflicting vehicles. The proposed algorithm is verified by the self-developed intelligent vehicle hardware simulation platform. The simulation results show that the PID method based on digital identification and positioning makes the intelligent vehicle obtain good system step response, can improve the disturbance tracking ability of intersection turning analysis, meet the requirements of turning control system, and reduce the complexity and randomness of parameter design, which is better than the traditional fuzzy control method.

LEARNING SOCIAL COMPLIANT MULTI-MODAL DISTRIBUTIONS OF HUMAN PATH IN CROWDS

Abstract. Long-term human path forecasting in crowds is critical for autonomous moving platforms (like autonomous driving cars and social robots) to avoid collision and make high-quality planning. It is not easy for prediction systems to successfully take into account social interactions and predict a distribution of future possible path in a highly interactive and dynamic circumstance. In this paper, we develop a data-driven model for long-term trajectory prediction, which naturally takes into account social interactions through a spatio-temporal graph representation and predicts multi-modes of future trajectories. Different from generative adversarial network (GAN) based models which generate samples and then provide distributions of samples, we use mixture density functions to describe human motion and intuitively map the distribution of future path with explicit densities. To prevent the model from collapsing into a single mode and truly capture the intrinsic multi-modality, we further use a Winner-Takes-All (WTA) loss instead of computing loss over all modes. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to capture the multi-modality of human motion and forecast the distributions of plausible futures in complex scenarios.

Space-Time Tree Search for Long-Term Trajectory Prediction

The pedestrian trajectory prediction is critical for autonomous driving, intelligent navigation, and abnormal behavior detection. With the booming of artificial intelligence (AI), many researchers have employed deep learning technologies to solve the pedestrian trajectory prediction problem and obtained relatively better performance in the short-term trajectory prediction. However, long-term trajectory prediction is still challenging to achieve high prediction accuracy. In this work, we propose a space-time tree search (STTS) method for long-term pedestrian trajectory prediction. Compared with existing methods only considering the problem from the space dimension, the proposed method formulates the trajectory prediction problem as a joint space-time tree search process by mapping the environment to a grid map. Since the human’s trajectory is relative to space and time dimensions, the trajectory prediction accuracy can be improved by the two dimensions. Then, a space-time reward trained neural network is employed to extract the pedestrian’s intent with both the scene image and the historical trajectory as input and outputs the prior search probabilities. Finally, the tree search can obtain the optimal predicted trajectory according to the prior probabilities, significantly improving the tree search efficiency. After testing, our proposed method can perform better than existing methods.