Meta-IRLSOT++: A meta-inverse reinforcement learning method for fast adaptation of trajectory prediction networks

Abstract

Recent research on pedestrian trajectory prediction based on deep learning has made significant progress. However, the previous methods do not deeply explore the relationship between scene information and trajectory. Moreover, the training model requires massive data and only targets specific scenes, so the prediction performance is poor when the new scene samples are limited. To solve the above problems, a meta-inverse reinforcement learning-based framework, dubbed Meta-IRLSOT++, is proposed in this work. IRLSOT++ improves the solid baseline IRLSOT, and the main contributions are as follows: (1) An inverse reinforcement learning framework is introduced to explore the trajectory–scene association to achieve task-level scene understanding, enhancing the correlation between trajectory and scene. The trajectory heat maps describing pedestrians dynamic characteristics are leveraged to align better the trajectory and scene semantic segmentation predicted by TopFormer. (2) A Transformer-based encoder–decoder network is proposed to fuse the trajectory and plan cues for better generating multi-modal trajectories with multi-head attention. The social graph attention and pseudo-oracle predictor are introduced to capture pedestrians’ social interactions and intent states, ameliorating trajectory prediction performance. (3) Meta-learning is leveraged to achieve collaborative training based on IRLSOT++ to improve the model generalization in new scenes, enabling fast adaptation of trajectory prediction. Experimental results on the Stanford Drone Dataset (SDD) indicate that IRLSOT++ can precisely forecast future trajectories by improving IRLSOT, decreasing Average Displacement Error/ Final Displacement Error (ADE/FDE) values from 9.66/13.05 to 8.36/12.28. Moreover, the meta-learning strategy quickly adapts IRLSOT++ to the new scene, achieving ADE/FDE values of 7.31/11.02 and 15.42/26.55 when Tpred is set to 12 and 24, respectively. Both quantitative and qualitative experimental results show that Meta-IRLSOT++ has accuracy and fast adaptability, which is beneficial for real-world trajectory prediction tasks.