Pedestrian Intention Research Articles

Predicting pedestrian intentions with multimodal IntentFormer: A Co-learning approach

The prediction of pedestrian crossing intention is a crucial task in the context of autonomous driving to ensure traffic safety and reduce the risk of accidents without human intervention. Nevertheless, the complexity of pedestrian behaviour, which is influenced by numerous contextual factors in conjunction with visual appearance cues and past trajectory, poses a significant challenge. Several state-of-the-art approaches have recently emerged that incorporate multiple modalities. Nonetheless, the suboptimal modality integration techniques in these approaches fail to capture the intricate intermodal relationships and robustly represent pedestrian-environment interactions in challenging scenarios. To address these issues, a novel Multimodal IntentFormer architecture is presented. It works with three transformer encoders {TEI,TEII,TEIII} which learn RGB, segmentation maps, and trajectory paths in a co-learning environment controlled by a Co-learning module. A novel Co-learning Adaptive Composite (CAC) loss function is also proposed, which penalizes different stages of the architecture, regularizes the model, and mitigates the risk of overfitting. Each encoder {TEη} applies the concept of the Multi-Head Shared Weight Attention (MHSWA) mechanism while learning three modalities in the proposed co-learning approach. The proposed architecture outperforms existing state-of-the-art approaches on benchmark datasets, PIE and JAAD, with 93 % and 92 % accuracy, respectively. Furthermore, extensive ablation studies demonstrate the efficiency and robustness of the architecture, even under varying Time-to-event (TTE) and observation lengths. The code is available at https://github.com/neha013/IntentFormer

Spatio-temporal deep learning framework for pedestrian intention prediction in urban traffic scenes

Pedestrian intent prediction is an essential task for ensuring the safety of pedestrians and vehicles on the road. This task involves predicting whether a pedestrian intends to cross a road or not based on their behavior and surrounding environment. Previous studies have explored feature-based machine learning and vision-based deep learning models for this task but these methods have limitations in capturing the global spatio-temporal context and fusing different features of data effectively. To address these issues, we propose a novel hybrid framework HSTGCN for pedestrian intent prediction that combines spatio-temporal graph convolutional neural networks (STGCN) and long short-term memory (LSTM) networks. The proposed framework utilizes the strengths of both models by fusing multiple features, including skeleton pose, trajectory, height, orientation, and ego-vehicle speed, to predict their intentions accurately. The framework’s performance have been evaluated on the JAAD benchmark dataset and the results show that it outperforms the state-of-the-art methods. The proposed framework has potential applications in developing intelligent transportation systems, autonomous vehicles, and pedestrian safety technologies. The utilization of multiple features can significantly improve the performance of the pedestrian intent prediction task.

Analysis of pedestrian second crossing behavior based on physics-informed neural networks

Pedestrian two-stage crossings are common at large, busy signalized intersections with long crosswalks and high traffic volumes. This design aims to address pedestrian operation and safety by allowing navigation in two stages, negotiating each traffic direction separately. Understanding crosswalk behavior, especially during bidirectional interactions, is essential. This paper presents a two-stage pedestrian crossing model based on Physics-Informed Neural Networks (PINNs), incorporating fluid dynamics equations to determine characteristics such as speed, density, acceleration, and Reynolds number during crossings. The study shows that PINNs outperform traditional deep learning methods in calculating and predicting pedestrian fluid properties, achieving a mean squared error as low as 10–8. The model effectively captures dynamic pedestrian flow characteristics and provides insights into pedestrian behavior impacts. The results are significant for designing pedestrian facilities to ensure comfort and optimizing signal timing to enhance mobility and safety. Additionally, these findings can aid autonomous vehicles in better understanding pedestrian intentions in intelligent transportation systems.

Dependent Hidden Markov Model for pedestrian intention prediction: considering Multivariate Interaction Force

Accurately recognizing and predicting pedestrian intentions is crucial for autonomous vehicle safety. However, existing prediction models often fail to comprehensively consider interactions between various traffic elements, resulting in suboptimal accuracy and robustness, especially in complex environments. To address this, we propose a pedestrian intention prediction model combining the Multivariate Interaction Force (MIF) model and a Dependent Hidden Markov Model (DE-HMM) for unsignalized midblock crossings. The MIF model captures dynamic interactions among pedestrians, vehicles, and the environment, while DE-HMM uses MIF data and pedestrian head orientation for predictions. Our model achieves 91.5% accuracy in recognizing crossing intentions, and 88.7% and 85.1% accuracy for predictions 0.5s and 1s ahead, respectively, outperforming current mainstream models and demonstrating strong robustness in special scenarios.

Multi-modal Pedestrian Trajectory Prediction based on Pedestrian Intention for Intelligent Vehicle

Multi-modal Pedestrian Trajectory Prediction based on Pedestrian Intention for Intelligent Vehicle

Pedestrian Trajectory Prediction Based on an Intention Randomness Influence Strategy

Pedestrian trajectory prediction is a key technical prerequisite for autonomous vehicle trajectory planning. However, a pedestrian is a changeable individual, and their intentions exhibit certain degrees of randomness and uncertainty, which leads to the issue that modeling only past trajectories does not enable the effective description of the random intentions and future trajectory directions of the pedestrian. Therefore, this paper proposes a flexible and embeddable stochastic intention vector construction strategy for modeling sudden pedestrian intention changes in real scenes and for better fitting the stochastic properties of pedestrian behaviors. First, we dynamically fuse historical trajectory information with random factors and construct an intention change probability based on the historical trajectory fitting errors of pedestrians, aiming to explicitly model the associated direction and velocity changes caused by random pedestrian intentions. Second, a new intention loss function is designed to guide the model to adaptively learn the probability of intention changes, which is used to dynamically describe pedestrian intention changes. Our proposed method is generalizable and can be applied as an embeddable module to any baseline pedestrian trajectory prediction method. The experimental results obtained on multiple large-scale public pedestrian trajectory prediction datasets demonstrate that our strategy achieves consistent performance improvements over different baselines.

DTDNet: Dynamic Target Driven Network for pedestrian trajectory prediction.

Predicting the trajectories of pedestrians is an important and difficult task for many applications, such as robot navigation and autonomous driving. Most of the existing methods believe that an accurate prediction of the pedestrian intention can improve the prediction quality. These works tend to predict a fixed destination coordinate as the agent intention and predict the future trajectory accordingly. However, in the process of moving, the intention of a pedestrian could be a definite location or a general direction and area, and may change dynamically with the changes of surrounding. Thus, regarding the agent intention as a fixed 2-d coordinate is insufficient to improve the future trajectory prediction. To address this problem, we propose Dynamic Target Driven Network for pedestrian trajectory prediction (DTDNet), which employs a multi-precision pedestrian intention analysis module to capture this dynamic. To ensure that this extracted feature contains comprehensive intention information, we design three sub-tasks: predicting coarse-precision endpoint coordinate, predicting fine-precision endpoint coordinate and scoring scene sub-regions. In addition, we propose a original multi-precision trajectory data extraction method to achieve multi-resolution representation of future intention and make it easier to extract local scene information. We compare our model with previous methods on two publicly available datasets (ETH-UCY and Stanford Drone Dataset). The experimental results show that our DTDNet achieves better trajectory prediction performance, and conducts better pedestrian intention feature representation.

The application of deep learning in autonomous driving

Autonomous driving technology is currently a globally prominent subject, with its relevance and impact readily apparent. Leveraging advanced sensors, sophisticated algorithms, and state-of-the-art computer vision techniques, autonomous vehicles can autonomously navigate, mitigate traffic accidents, and alleviate urban congestion. Furthermore, this technology is poised to accelerate innovation within the automotive sector, drive industrial advancement, and enhance people's convenience and safety in their travel experiences. The importance of autonomous driving technology is not only reflected in its own advantages, but also in its ability to lead the development direction of intelligent transportation in the future, helping to promote the development of intelligent transportation systems, realize the intelligent interconnection between vehicles and vehicles, vehicles and road infrastructure, and further enhance the safety and convenience of traffic travel. Therefore, autonomous driving technology is a significant innovation that will bring a better and more convenient future for mankind. In this study, a method based on sensor internal and external parameter calibration transformation matrix, road target detection algorithm, automatic vehicle detection method, pedestrian intention prediction technology and automatic pedestrian recognition system are analyzed, which are applied to object detection and obstacle avoidance. This article provides a good overview of the field of autonomous driving.

Small group pedestrian crossing behaviour prediction using temporal angular 2D skeletal pose

A pedestrian is classified as a Vulnerable Road User (VRU) because they do not have the protective equipment that would make them fatal if they were involved in an accident. An accident can happen while a pedestrian is on the road, especially when crossing the road. To ensure pedestrian safety, it is necessary to understand and predict pedestrian behaviour when crossing the road. We propose pedestrian intention prediction using a 2D pose estimation approach with temporal angle as a feature. Based on visual observation of the Joint Attention in Autonomous Driving (JAAD) dataset, we found that pedestrians tend to walk together in small groups while waiting to cross, and then this group is disbanded on the opposite side of the road. Thus, we propose to perform prediction with small group of pedestrians, based on pedestrian statistical data, we define a small group of pedestrians as consisting of 4 pedestrians. Another problem raised is 2D pose estimation is processing each pedestrian index individually, which creates ambiguous pedestrian index in consecutive frame. We propose Multi Input Single Output (MISO), which has capabilities to process multiple pedestrians together, and use summation layer at the end of the model to solve the ambiguous pedestrian index problem without performing tracking on each pedestrian. The performance of our proposed model achieves model accuracy of 0.9306 with prediction performance of 0.8317.

Pedestrian intention estimation and trajectory prediction based on data and knowledge‐driven method

AbstractWith the development of deep learning technology, the problem of data‐driven trajectory prediction and intention recognition has been widely studied. However, the pedestrian trajectory prediction and intention recognition methods based solely on data‐driven have weak data description ability and black‐box characteristics, which cannot reason about pedestrian crossing intention and predict pedestrian crossing trajectory as humans do. To address the above problems, the authors proposed a data and knowledge‐driven pedestrian intention estimation and trajectory prediction method by imitating human cognitive mechanisms. In the pedestrian intention inference process, the authors adopted the knowledge‐driven method. As a first step, the authors built a knowledge graph of pedestrian crossing scenes, and then paired it with a Bayesian network to estimate pedestrian crossing intentions. In the pedestrian trajectory prediction process, the authors used a data‐driven approach, combining pedestrian crossing trajectory features and knowledge‐based pedestrian intentions. Experiments show that all evaluation metrics of pedestrian trajectory prediction were improved after adding pedestrian intentions obtained by knowledge‐driven.

Visual–Motion–Interaction-Guided Pedestrian Intention Prediction Framework

Visual–Motion–Interaction-Guided Pedestrian Intention Prediction Framework

Integration of Wearables and Wireless Technologies to Improve the Interaction between Disabled Vulnerable Road Users and Self-Driving Cars

The auto industry is accelerating, and self-driving cars are becoming a reality. However, the acceptance of such cars will depend on their social and environmental integration into a road traffic ecosystem comprising vehicles, motorcycles, bicycles, and pedestrians. One of the most vulnerable groups within the road ecosystem is pedestrians. Assistive technology focuses on ensuring functional independence for people with disabilities. However, little effort has been devoted to exploring possible interaction mechanisms between pedestrians with disabilities and self-driving cars. This paper analyzes how self-driving cars and disabled pedestrians should interact in a traffic ecosystem supported by wearable devices for pedestrians to feel safer and more comfortable. We define the concept of an Assistive Self-driving Car (ASC). We describe a set of procedures to identify people with disabilities using an IEEE 802.11p-based device and a group of messages to express the intentions of disabled pedestrians to self-driving cars. This interaction provides disabled pedestrians with increased safety and confidence in performing tasks such as crossing the street. Finally, we discuss strategies for alerting disabled pedestrians to potential hazards within the road ecosystem.

Pedestrian Crossing Intention Prediction Model Considering Social Interaction between Multi-Pedestrians and Multi-Vehicles

Accurate pedestrian crossing intention prediction is critical for autonomous vehicles and advanced driver assistance systems. In multi-pedestrian and multi-vehicle interaction scenes, the social interaction between pedestrians and other traffic participants is ubiquitous, which affects pedestrian crossing decisions and the accuracy of prediction. However, previous studies on pedestrian crossing intention lack comprehensive consideration and mathematical modeling of the social interaction. We propose a “social interaction force” (SIF) to identify and quantify social interaction behaviors and combine the hidden Markov model (HMM) to predict pedestrian crossing intentions 1.0 s ahead. Firstly, a large dataset of pedestrian-vehicle interaction samples is collected from two views, and high-dimensional features are extracted for pedestrian intention prediction. Next, the concept of SIF is proposed to quantitatively characterize the influence of other pedestrians and vehicles on pedestrian crossing decisions, including “pedestrian interaction force” and “pedestrian-vehicle interaction force.” Finally, SIF, pedestrian features, and road structure features are input into HMM. Sliding time windows are applied to the HMM to achieve dynamic prediction of pedestrian intention sequences. Experimental results show that the recognition accuracy of the proposed model is 0.976, and the accuracy of 1.0 s ahead prediction is 0.932 with guaranteed prediction speed. The proposed model performance is superior to that of the most prevalent models developed thus far.

Understanding Pedestrian Trajectory Prediction from the Planning Perspective

Pedestrians are essential component in traffic, and with the advent of intelligent transportation systems, conflicts over right‐of‐way between pedestrians and other traffic participants have become prevalent. Predicting pedestrian intentions and motions is crucial for autonomous driving and intelligent transportation. However, pedestrian prediction faces challenges such as the variability of pedestrian intentions and the flexibility of their motions, as well as the need for highly generalization capability and lightweight modeling on autonomous vehicles with limited computing power. To face these challenge, this paper proposes a Social Artificial Potential Field (Social‐APF) method for pedestrian trajectory prediction. The method combines target intention feature and surrounding obstacles features. We test the performance of Social‐APF on several public datasets. The results demonstrate that the Social‐APF method outperforms state‐of‐the‐art approaches on the public datasets in both lightweight and model accuracy. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Multi-attention network for pedestrian intention prediction based on spatio-temporal feature fusion

An essential prerequisite for autonomous vehicles deploying in urban scenarios is the ability to accurately recognize the behavioral intentions of pedestrians and other vulnerable road users and take measures to ensure their safety. In this paper, a spatial-temporal feature fusion-based multi-attention network (STFF-MANet) is designed to predict pedestrian crossing intention. Pedestrian information, vehicle information, scene context, and optical flow are extracted from continuous image sequences as feature sources. A lightweight 3D convolutional network is designed to extract temporal features from optical flow. Construct a spatial encoding module to extract the spatial features from the context. Pedestrian motion information are re-encoded using a collection of gated recurrent units. The final network structure is created through ablation research, which introduces attention mechanisms into the network to merge pedestrian motion features and spatio-temporal features. The efficiency of the suggested strategy is demonstrated by comparison experiments on the datasets JAAD and PIE. On the JAAD dataset, the intent recognition accuracy is 9% more accurate than the existing techniques.

TrEP: Transformer-Based Evidential Prediction for Pedestrian Intention with Uncertainty

With rapid development in hardware (sensors and processors) and AI algorithms, automated driving techniques have entered the public’s daily life and achieved great success in supporting human driving performance. However, due to the high contextual variations and temporal dynamics in pedestrian behaviors, the interaction between autonomous-driving cars and pedestrians remains challenging, impeding the development of fully autonomous driving systems. This paper focuses on predicting pedestrian intention with a novel transformer-based evidential prediction (TrEP) algorithm. We develop a transformer module towards the temporal correlations among the input features within pedestrian video sequences and a deep evidential learning model to capture the AI uncertainty under scene complexities. Experimental results on three popular pedestrian intent benchmarks have verified the effectiveness of our proposed model over the state-of-the-art. The algorithm performance can be further boosted by controlling the uncertainty level. We systematically compare human disagreements with AI uncertainty to further evaluate AI performance in confusing scenes. The code is released at https://github.com/zzmonlyyou/TrEP.git.

Recognition Assistance Interface for Human-Automation Cooperation in Pedestrian Risk Prediction

<div>Autonomous driving systems (ADS) have been widely tested in real-world environments with operators who must monitor and intervene due to remaining technical challenges. However, intervention methods that require operators to take over control of the vehicle involve many drawbacks related to human performance. ADS consist of recognition, decision, and control modules. The latter two phases are dependent on the recognition phase, which still struggles with tasks involving the prediction of human behavior, such as pedestrian risk prediction. As an alternative to full automation of the recognition task, cooperative recognition approaches utilize the human operator to assist the automated system in performing challenging recognition tasks, using a recognition assistance interface to realize human-machine cooperation. In this study, we propose a recognition assistance interface for cooperative recognition in order to achieve safer and more efficient driving through improved human-automation cooperation. A simulator experiment with 18 participants is conducted to evaluate our recognition assistance interface in comparison with a conventional control intervention, in terms of driving safety, efficiency, and usability. Recognition of pedestrian crossing intention is selected for the cooperation task, and driving scenarios in which the automated system cannot reliably recognize the crossing intentions of pedestrians at non-signalized locations are selected as the driving scenario. Statistical analysis of our experimental results reveals that the proposed recognition assistance interface allowed more accurate operator intervention, was easier to use, and achieved more stable vehicle control than the control intervention. We also found that sharing the recognition information of the automated driving system with operators could divide their attention, impairing intervention performance. Our experimental results suggest that the unifying presentation of the system recognition information and the operator’s manipulation target on the touchscreen of the user interface addresses this problem.</div>

Recognition and Prediction of Pedestrian Hazardous Crossing Intentions in Visual Field Obstruction Areas Based on IPVO-LSTM

Pedestrians who suddenly cross the street from within the blind spot of a vehicle’s field of view can pose a significant threat to traffic safety. The dangerous pedestrian crossing intentions in view-obscured scenarios have not received as much attention as the prediction of pedestrian crossing intentions. In this paper, we present a method for recognizing and predicting the dangerous crossing intention of pedestrians in a view-obscured region based on the interference, pose, velocity observation–long short-term memory (IPVO-LSTM) algorithm from a road-based view. In the first step, the road-based camera captures the pedestrian’s image. Then, we construct a pedestrian interference state feature module, pedestrian three-dimensional pose feature module, pedestrian velocity feature module, and pedestrian blind observation state feature module and extract the corresponding features of the studied pedestrians. Finally, the pedestrian hazard crossing intention prediction module based on a feature-fused LSTM (ff-LSTM) and attention mechanism is used to fuse and process the above features in a cell state process to recognize and predict the pedestrian hazard crossing intention in the blind visual area. Experiments are compared with current common algorithms in terms of the input parameter selection, intention recognition algorithm, and intention prediction time range, and the experimental results validate our state-of-the-art method.

Active Road User Interactions With Autonomous Vehicles: Proactive Safety Assessment

This study aims to conduct a thorough assessment of pedestrian and cyclist safety in autonomous vehicle (AV) environments. To that end, the study utilized AV sensor data of over 1,500 driving hours from five sources in Canada, the United States, and Singapore. The sensor data were used to extract conflicts between AVs and active road users. The conflicts were then processed to develop accurate estimates of AV collisions involving pedestrians and cyclists based on the extreme value theory. Further in-depth assessments were conducted on the identified conflicts, by type and location, to highlight potential issues leading to risky conflicts. The results showed that the total number of predicted AV collisions involving active road users was 2.17 collisions per million AV kilometers travelled. Collisions involving pedestrians were slightly higher than those involving cyclists. Also, collisions in clear weather conditions slightly exceeded collisions in adverse weather conditions, although the difference was not statistically significant. The relative risk of collisions was developed for both pedestrian and cyclist conflicts per AV movement type. The results showed that for pedestrians, interactions with right-turning AVs are the riskiest, while interactions with left-turning AVs are the riskiest for cyclists. A thorough analysis of conflicts revealed many issues, including a higher tendency for pedestrian violations when interacting with AVs, aggressive AV behavior (particularly when interacting with pedestrians while making a right turn), AVs struggling to predict the path of cyclists (mainly because of cyclist violations), and AVs failing to interpret pedestrian intentions in some cases.

Texting and crossing: An extended theory of planned behaviour to model the psychological and demographic factors related to pedestrians' use of cell phone for texting at crosswalks in developing country

The distraction of road users is one of the leading causes of road crashes. In general, distraction in road crashes is often associated with only driving, not walking, however, several studies have highlighted distracted walking as a major cause for road crashes and have also examined distracted walking behaviour and its causes, but there is a paucity of such kind of literature in the context of developing or low-income countries, such as India. This study sought to fill this gap by examining factors that influence pedestrians' use of cell phones for texting while crossing the roads in the city of Bhopal, India. In the present study analysis of psychological factors and socio-economic factors that contribute to distracted walking/crossing was analysed using an extended version of the theory of planned behaviour (TPB). Using confirmatory factor analysis (CFA) and structural equation modelling (SEM), this study confirmed the validity of items under each factor of the TPB and then employed SEM to investigate the relationship between the latent variables (attitude, subjective norm, and perceived behaviour control), and demographic characteristics (age, income, and gender) of pedestrians with their intention to text while crossing the street. The results obtained from SEM indicated that the intention to use cell phones for texting at crosswalks was negatively impacted by age, and positively impacted by income level. A significant influence of attitudes, subjective norms, and perceived behaviour control was found on the intention to use cell phones. Perceived behaviour control (PBC) was found as the most influential factor for predicting pedestrian intentions to use cell phones, followed by Subjective norms (SN). In addition, perceived behaviour control and the intention to use a cell phone also significantly affected crossing behaviour. The findings from the present study can significantly contribute to enhancing pedestrian safety in transportation research, and a better understanding of the factors contributing to pedestrian fatalities could lead towards safe system approach.