Intelligent scheduling of discrete automated production line via deep reinforcement learning

Abstract

The reinforcement learning (RL) is being used for scheduling to improve the adaptability and flexibility of an automated production line. However, the existing methods only consider processing time certain and known and ignore production line layouts and transfer unit, such as robots. This paper introduces deep RL to schedule an automated production line, avoiding manually extracted features and overcoming the lack of structured data sets. Firstly, we present a state modelling method in discrete automated production lines, which is suitable for linear, parallel and re-entrant production lines of multiple processing units. Secondly, we propose an intelligent scheduling algorithm based on deep RL for scheduling automated production lines. The algorithm establishes a discrete-event simulation environment for deep RL, solving the confliction of advancing transferring time and the most recent event time. Finally, we apply the intelligent scheduling algorithm into scheduling linear, parallel and re-entrant automated production lines. The experiment shows that our scheduling strategy can achieve competitive performance to the heuristic scheduling methods and maintains stable convergence and robustness under processing time randomness.