Parallel Machine Workshop Scheduling Using the Integration of Proximal Policy Optimization Training and Monte Carlo Tree Search

Abstract

In this paper, a model called AlphaSchedule based on the ''search + training" framework is proposed to solve the scheduling problem of minimizing weighted tardiness in parallel machine workshop. A Markov Decision Process (MDP) model was established based on the parallel machine problem, and we designed the state features of the workshop, the action space for job scheduling, and the reward function equivalent to the scheduling objective. The AlphaSchedule model uses proximal policy optimization (PPO) algorithm to train a scheduling Resnet network, and integrates the trained scheduling network with an improved Monte Carlo tree search (MCTS) algorithm. Multiple comparison experiments in the simulation workshop show that the AlphaSchedule model has a better scheduling performance than genetic algorithm (GA), particle swarm optimization (PSO), and pure MCTS algorithm. Additional experiments show that the PPO algorithm in the AlphaSchedule model has higher training efficiency than the Monte Carlo tree training method in the AlphaGo Zero model.