Deep reinforcement learning for collision avoidance of autonomous ships in inland river

Abstract

Due to the characteristics of large inertia, large time delay, and nonlinearity in ship motion, the maneuverability of ships is an important issue related to the safety of ship navigation. The manipulation of ships significantly affects the safety of ship navigation, and its importance will gradually increase with the development of autonomous ships. Collision accidents are common due to the complexity of inland waterways and density of the ships. Herein, an autonomous learning framework with deep reinforcement learning (DRL) is constructed for autonomous driving tasks. The state space, action space, reward function and neural network structure of DRL are designed based on the ship’s maneuvering characteristics and control requirements. A DDPG algorithm is then used to implement the controller. Finally, some representative route segments of the inland waterway is selected for simulation research based on the virtual simulation environment. The designed DRL controller can quickly converge from the training and learning process to meet the control requirements. By comparing with the previous experimental results, the effectiveness of this algorithm is verified. Therefore, this study provides a reference for future research on the collision avoidance technology of autonomous ships in inland rivers.