Vessel-following model for inland waterways based on deep reinforcement learning

Abstract

With the growth of traffic on inland waterways, autonomous driving technologies for vessels will gain increasing significance to ensure traffic flow and safety. Inspired by car-following models for road traffic, which demonstrated their strength to reduce stop-and-go waves and increase efficiency and safety, we propose a vessel-following model for inland waterways based on deep reinforcement learning (RL). Our model is trained under consideration of realistic vessel dynamics and environmental influences, such as varying stream velocity and river profile, and with a reward function favoring observed following behavior and comfort. Aiming at high generalization capabilities, we propose a training environment that uses stochastic processes to model leading the trajectory and river dynamics. Our model demonstrated safe and comfortable driving in different unseen scenarios, including realistic vessel-following on the Middle Rhine. In comparison with an existing model, our model was able to early anticipate safety–critical situations, resulting in higher safety while maintaining comparable efficiency and comfort. In further experiments, the proposed approach demonstrated its potential to dampen traffic oscillations and reduce congestion by using a sequence of followers.