Acquisition of a Gymnast-Like Robotic Giant-Swing Motion by Q-Learning and Improvement of the Repeatability

Abstract

This paper proposes an application of Q-learning to a compact humanoid robot, aiming at acquisition of a gymnastic swinging even through the Markov property may not be guaranteed in such dynamic motions. As for this, several studies have relied on information from the robotic models or multiple controllers, but very few studies have tried Q-learning of human-like swing motion without preliminary knowledge. We avoid this Markov property problem by embedding the dynamic information in a robotic state space and averaging action-value functions. In this study, Q-learning is executed with a dynamic simulator based on a real humanoid robot with 5 degrees of freedom (DOF) and we verify the learning effectiveness by actually applying the learning results to the real robot. The particularly significant point in our Q-learning is that preliminary information is eliminated as far as possible; only the reward and current robotic state are available. The key factor in robotic giant-swing motion is discussed by examining effects of various rewards on the robotic performance. In addition, we argue a method for improving the repeatability and duration until reaching giant-swing motion. Finally, this paper demonstrates an attractive robotic giant-swing motion generated by only the environmental interaction.