Hierarchical Policy Learning With Demonstration Learning for Robotic Multiple Peg-in-Hole Assembly Tasks

Abstract

The force-based control algorithm of robotic multiple peg-in-hole assembly is a challenge. Forthe difficulty of low adapt-ability of model-based control algorithms and low learning effi-ciency of model-free control algorithms, a goal-based hierarchical policy learning (HPL) algorithm that combines conventional con-trol algorithm and demonstration learning algorithm is proposed to learn the assembly skill. Firstly, the goal-based HPL algorithm adds goal as a new variable to the action value function. Multiple states reached in each episode are randomly selected as sub-goals to improve the distributionof positive rewards. Secondly, an initial policy that combines conventional control algorithm and demon-stration learning algorithm is designed. The combined coefficient of these two algorithms is learned by HPL algorithm. Finally, a conical surface is used to compute the forces and moments of sim-plified assembly simulation model. Our algorithm is well imple-mented in both simulation and real-world environments. The ex-perimental results verify the effectiveness of the proposed method.