A parallelized environmental-sensing and multi-tasks model for intelligent marine structure control in ocean waves coupling deep reinforcement learning and computational fluid dynamics

Abstract

In addressing the active control challenges of marine structures in ocean waves, a coupling model is proposed combining computational fluid dynamics (CFD) and deep reinforcement learning (DRL). Following the Markov decision process (MDP), the proposed DRL-CFD model treats the wave fields and simplified marine structures as the environment and the agent, respectively. The CFD component utilizes the PIMPLE algorithm to solve the Navier–Stokes equations, in which the free surface is reconstructed using the volume of fluid method. The DRL component utilizes the Soft Actor-Critic algorithm to realize the MDP between marine structures and the wave fields. Three simulation cases with different control purposes are conducted to show the effectiveness of the DRL–CFD coupling model, including the active controls for wave energy absorption, attenuation, and structure heave compensation. Comparative analyses with passive (resistive) control are performed, demonstrating the advantages of the DRL–CFD coupling model. The results confirm that the proposed coupling model enables the marine structure to observe the wave environment and generate effective active control strategies for different purposes. This suggests that the model has the potential to address various active control challenges of marine structures in ocean waves, while being capable of environmental sensing and handling multiple tasks simultaneously.