Design of Real-time Pedestrian Trajectory Prediction System based on Jetson Xavier

Abstract

This paper presents a vehicle-mounted real-time pedestrian trajectory prediction system based on the embedded device Jetson Xavier. It achieves low-cost real-time pedestrian trajectory prediction using only the front camera of the vehicle. Firstly, pedestrian detection and tracking are implemented based on YoLoV7, while also estimating pedestrian poses and optical flow to provide multiple information sequences for the trajectory prediction network. Secondly, the pedestrian trajectory algorithm from a driver's perspective is studied, and a trajectory prediction algorithm that considers pedestrian pose, optical flow, and trajectory information is proposed. A novel multi-information fusion network is designed to better integrate multiple features. The algorithm is tested on the JAAD and PIE datasets, and the displacement errors are reduced by 6.35% and 3.28%, respectively, compared to BiTraP. Finally, the algorithm is ported to the embedded device Xavier and installed on a simulated vehicle for testing. By predicting pedestrian future trajectories based on pedestrian detection, collisions can be avoided in advance, improving the safety of autonomous driving. The proposed system has significant practical value.