Pedestrian potentially dangerous behaviour prediction based on attention‐long‐short‐term memory with egocentric vision

Abstract

AbstractThis paper develops a pedestrian potentially dangerous behaviour prediction method based on attention‐long‐short‐term memory (Attention‐LSTM) architecture to predict pedestrian trajectory and intention for the unexpected pedestrian crossing accident avoidance. To extract the road scene information for short periods of time, and improve the accuracy of subsequent intention inference and trajectory prediction, the panoramic segmentation is used to extrapolate pedestrian instances and segment areas of the environment. Next, an encoder–decoder framework based on Attention‐LSTM model is proposed to infer a pedestrian's intention to run or walk out into oncoming traffic straight and to predict the future trajectory. The proposed network involves two parts: temporal feature encoder and multi‐task decoder. The temporal feature encoder is mainly used to selectively emphasize the temporal features using attention mechanism, and then LSTM is employed for its encoding. In the multi‐task decoder, a multi‐head self‐attention mechanism and LSTM are used to forecast the pedestrians’ intention and future trajectory, respectively. Extensive experiments on pedestrian intention estimation (PIE) datasets demonstrate that the authors’ proposed approach surpasses prior studies in terms of prediction accuracy in trajectory and intention. This study can not only effectively avoid serious accidents caused by illegal road crossing but also achieve early warning for collision avoidance.