Optimization of Central Pattern Generator-Based Locomotion Controller for Fish Robot Using Deep Deterministic Policy Gradient

Abstract

AbstractThis paper presents the optimization of a CPG-based locomotion controller for a fish robot using Deep Deterministic Policy Gradient (DDPG). Firstly, the rhythm of swimming of an elongated undulating fin-like black Knife fish is generated by Central Pattern Generator (CPG). In the CPG network, the Hopf oscillators are employed to provide the rhythmical output and ensure continuous sinusoidal oscillation even when the CPG parameters are abruptly changed. The smooth transition output of the CPG is dependent on an intrinsic parameter of the oscillator called the convergence speed. This parameter is optimized by a combination of Deep Q-Network (DQN) and Policy Gradient (PG), which overcomes the drawback of traditional DQN, such as providing stable learnings to adapt specifically to dynamic environments. The simulation results demonstrate that the convergence speed of the modified CPG network based on DDPG is improved by about 2.2%. It also indicates that the rhythmical output of the CPG integrated with the DDPG optimizer can provide higher accuracy of oscillatory amplitude (about 1,6%) than do the traditional DQN, leading to high efficiency in controlling the swimming gait of the robotic fish.KeywordsCentral pattern generatorDeep deterministic policy gradient