Reinforcement Learning-based Scheduling of a Job-Shop Process with Distributedly Controlled Robotic Manipulators for Transport Operations

Abstract

Job-shop scheduling problems are important in the industrial context to achieve high machine utilization. Heuristics offer a possibility to solve these problems with moderate computational effort. However, they might be associated with a high development effort and generalization to other tasks is difficult. We use a reinforcement learning approach (deep Q-learning) to solve a job-shop problem in our production environment. A production process is considered where jobs are transported to allocated stations by two collaborative robots. To this end, a learning environment and a simulation environment are developed to evaluate the feasibility of an obtained schedule. The results are compared to a First In - First Out heuristic. The main objective is to consider the motion of the robots and to avoid collisions without losing unnecessary time. First, a fixed scheduling problem is analyzed to verify that a feasible solution can be obtained. Second, arbitrary instances of the scheduling problem are solved. The presented method leads to feasible schedules. An increased training and a more stable convergence process are necessary for an efficient use.