Social Learning with Actor–Critic for dynamic grasping of underwater robots via digital twins

Abstract

Dynamic grasping is a crucial technology in the field of underwater robotics, playing an essential role in executing complex tasks. However, traditional control methods encounter challenges in achieving an efficient and stable dynamic grasping process due to the complexities and uncertainties of underwater environments. This paper introduces a novel black-box intelligent optimization algorithm named Social Learning with Actor–Critic (SLAC) for the dynamic grasping of underwater robots. The core architecture of SLAC is based on the integration of two key algorithms: Intelligent Social Learning (ISL) for intelligent optimization and Soft Actor–Critic (SAC) for reinforcement learning. ISL enhances SAC by supplying a larger number of transitions, whereas SAC improves ISL with more effective strategies. These algorithms interact synergistically, augmenting their respective strengths throughout the learning process. To evaluate SLAC’s performance, a comparison is made with six state-of-the-art methods across eight continuous control benchmark cases. The results highlight SLAC’s exceptional learning capability and performance benefits. Furthermore, virtual reality software for the underwater robot and a corresponding digital twin system have been developed. The SLAC algorithm is trained in the digital twin environment before its application in the actual underwater setting. Through interactive training and iterative learning, both simulated and experimental results demonstrate the robot’s proficiency in achieving efficient and stable dynamic grasping, effectively adapting to various underwater environments’ variations and complexities.