Dynamic trajectory planning for ships in dense environment using collision grid with deep reinforcement learning

Abstract

In areas with high maritime traffic, ship safety is of utmost importance when validating autonomous navigation models. While exact methods exist for specific situations, they are inadequate in a global context. This study employs an approximate deep reinforcement learning method to solve the navigation problem in a dense environment with numerous static and moving obstacles. Our model prioritizes ship integrity by enabling the agent to dynamically adapt its kinematics to its surroundings to reach a designated goal without colliding with obstacles. To achieve this, we incorporate collision grids in the form of danger zones as input to our model and train it using the proximal policy optimization algorithm. Additionally, we propose implementing the International Regulations for Preventing Collisions at Sea (COLREGs) in the collision grid as these navigation rules are necessary to obtain realistic behavior of an autonomous agent. The agent’s performance is evaluated on a set of randomly generated scenarios operating in an environment complexity similar to the one used during training. These tests demonstrate that this type of data structure allows a trained agent to navigate in a dense environment while adhering to the COLREGs with a success rate of 94.69%.