A path planning strategy for marine vehicles based on deep reinforcement learning and data-driven dynamic flow fields prediction

Abstract

This paper presents a strategy for planning a path of a marine vehicle in dynamic flow fields. This strategy composes of two modules: deep reinforcement learning based path planning and dynamic mode decomposition (DMD) based flow fields prediction. The path planning module employs the deep reinforcement learning algorithm of proximal policy optimization (PPO) to implement the time-optimal path planning of a marine vehicle in predicted spatially-temporally dynamic flow fields, where the long short-term memory (LSTM) is introduced to address the partially observable issue. The objective of the flow prediction module is to provide the path planning module with predicted dynamic flow fields. In the flow prediction module, the data-driven method of DMD is used to learn the low-dimensional model of flow dynamics and make future predictions. And a network of marine vehicles with flow sensing capability are adopted to generate data of flow fields for the on-line DMD learning and prediction, where their flow sensing locations are optimized by the deep reinforcement learning algorithm of deep-Q learning with the aim at minimizing the reconstruction error of the flow field with the sparse in-situ point flow observations by the swarm of marine vehicles. The strategy is implemented in computer simulations, where the flow data outputted by a numerical ocean model is utilized to test the strategy. The simulation results demonstrate the performance of the proposed strategy.