Local connection reinforcement learning method for efficient robotic peg-in-hole assembly

Abstract

Traditional robotic peg-in-hole assembly methods rely on complex contact state analysis. Reinforcement learning (RL) is gradually becoming a preferred method of controlling robotic peg-in-hole assembly tasks. However, the training process of RL is quite time-consuming because RL methods are always globally connected, which means all state components are assumed to be the input of policies for all action components, thus increasing action space and state space to be explored. In this paper, we first define continuous space serialized Shapley value (CS3) and construct a connection graph to clarify the correlativity of action components on state components. Then we propose a local connection reinforcement learning (LCRL) method based on the connection graph, which eliminates the influence of irrelevant state components on the selection of action components. The simulation and experiment results demonstrate that the LCRL method can achieve the same average reward as the traditional RL method in only 49% of episodes. In the final episode, the LCRL method's final reward is 35% higher than that of the traditional RL method, which guarantees the rapidity and stability of the assembly process.