Reinforcement Learning for Robotic Assembly Using Non-Diagonal Stiffness Matrix

Abstract

Contact-rich tasks, wherein multiple contact transitions occur in a series of operations, have been extensively studied for task automation. Precision assembly, a typical example of contact-rich tasks, requires high time constants to cope with the change in contact state. Therefore, this letter proposes a local trajectory planning method for precision assembly with high time constants. Because the non-diagonal component of a stiffness matrix can induce motion at high sampling frequencies, we use this concept to design a stiffness matrix to guide the motion of an object and propose a method to control it. We introduce reinforcement learning (RL) for the selection of the stiffness matrix because the relationship between the desired direction and the sensor response is difficult to model. An architecture with various sampling rates for RL and admittance control has the advantage of rapid response owing to a high time constant of the local trajectory modification. The effectiveness of the method is verified experimentally on two contact-rich tasks: inserting a peg into a hole and inserting a gear. Using the proposed method, the average total time needed to insert the peg in the hole is 1.64 s, which is less than half the time reported by the best of the existing state of the art studies.