Continual Pedestrian Trajectory Learning With Social Generative Replay

Abstract

Learning to predict the trajectories of pedestrians is essential for improving safety and efficiency of mobile robots. The prediction is challenging since the robot needs to operate in multiple environments in which the motion patterns of pedestrians are different between environments. Existing pedestrian trajectory prediction models heavily rely on the availability of representative data samples during training. In the presence of additional training data from a new environment, these models must be retrained on all datasets to avoid <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">catastrophic forgetting</i> of the knowledge obtained from the already supported environments. In this paper, we address this catastrophic forgetting problem in the context of learning to predict the trajectories of pedestrians. We propose a pseudo-rehearsal approach based on a novel Generative Replay (GR) model, referred to as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Social-GR</i> . The proposed method is consistent with crowd motion patterns and is free of any explicit reference to past experiences. To demonstrate the problem of catastrophic forgetting and evaluate our solution, we develop the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Continual Trajectory Prediction Benchmark</i> , which consists of four tasks, each representing a real-world pedestrian trajectory dataset from a different environment. By conducting several experiments, we show that our proposed Social-GR approach significantly outperforms other continual learning methods that depend on explicit experience replay, including the state-of-the-art conditional-GR model. We further illustrate the robustness of our proposed approach to mitigating catastrophic forgetting by switching the order of environments and employing a more complex prediction model.