Autonomous Skill Learning of Water Polo Ball Heading for a Robotic Fish: Curriculum and Verification

Abstract

This paper concerns the autonomous skill learning of water polo ball heading for a robotic fish in highly dynamic aquatic environments via Multi-stage Serial-Parallel Curriculum Learning (MSPCL). First, a radio-controlled robotic fish with hybrid fin propulsion is presented. Moreover, the water polo ball heading task is decomposed into two sequential subtasks: preparation and shooting. In order to cope with the multi-stage complex tasks, based on the easy-to-hard curriculum learning strategy, a novel MSPCL framework is proposed for the robotic fish to learn complex skills, which mainly includes curriculum scheduler, difficulty criterion, serial-parallel curriculum generation and performance measure modules. Furthermore, under the MSPCL framework, Soft Actor-Critic (SAC) algorithm is utilized for training the policy network. Therefore, the robotic fish can learn how to head the water polo ball via the proposed MSPCL method. Comparative simulations are carried out to verify the effectiveness of the MSPCL framework. Finally, the proposed MSPCL method is applied to a physical robotic fish named RoboDact. Swimming pool experiments of water polo ball heading of the RoboDact are conducted to demonstrate the validity and robustness of the proposed method.