Reinforcement Learning Enabled Self-Homing of Industrial Robotic Manipulators in Manufacturing

Abstract

Industrial robotics has been playing a major role in manufacturing across all types of industries. One common task of robotic cells in manufacturing is called homing, a step that enables a robotic arm to return to its initial / home position (HPos) from anywhere in a robotic cell, without collision or experiencing robot singularities while respecting its joint limits. In almost all industrial robotic cells, an operation cycle starts from, and ends to HPos. The home position also works as a safe state for a cycle to restart when an alarm or fault occurs within the cell. When an alarm occurs, the robot configuration in the cell is unpredictable, thus challenging to bring the robot, autonomously and with safety at HPos and restart the operation. This paper presents a non-vision, reinforcement learning-based approach of a parallel-agent setting to enable self-homing capability in industrial robotic cells, eliminating the need of manual programming of robot manipulators. This approach assumes the sensing of an unknown robotic cell environment pre-encoded in the state definition so that the policies learned can be transferred without further training. The agents are trained in a simulation environment generated by the mechanical design of an actual robotic cell to increase the accuracy of mapping the real environment to the simulated one. The approach explores the impact of certain curriculum on the agent’s learning and evaluates two choices, compared to a non-curriculum baseline. A parallel-agent, multi-process training setting is employed to enhance performance in exploring the state space, where experiences are shared among the agents via shared memory. Upon deployment, all agents are involved with their respective policies in a collective manner. The approach has been demonstrated in simulated industrial robotic cells, and it has been shown that the policies derived in simulation are transferable to a corresponding real industrial robotic cells, and are generalizable to other robotic systems in manufacturing settings.