Position and Attitude Tracking Control of a Biomimetic Underwater Vehicle via Deep Reinforcement Learning

Abstract

This article addresses a deep reinforcement learning (DRL) control method of position and attitude tracking for a biomimetic underwater vehicle (BUV). The BUV is actuated by two biomimetic propulsors. Each propulsor has a thick and flexible fin, which is manipulated by 12 short fin rays and can undulate in multiple wave patterns for propulsion. To achieve position and attitude tracking control on the BUV, a periodic dynamics-reparameterized soft actor-critic (SAC) algorithm is proposed. In detail, the algorithm uses the DRL method of SAC to train the controller by interacting with a simulated BUV, which is based on the propulsion model of the undulatory fin. Considering that the simulated environment may be inaccurate when compared with the real environment, some specially designed tricks are proposed. Simulations and experiments are conducted to prove the effectiveness and robustness of the proposed controller.