Sequence generation for multi-task scheduling in cloud manufacturing with deep reinforcement learning

Abstract

Cloud manufacturing is a manufacturing model that aims to deliver on-demand manufacturing services to consumers. Scheduling is an important problem that needs to be addressed carefully and effectively for cloud manufacturing to achieve that aim. Cloud manufacturing allows consumers to submit their requirements to the cloud platform simultaneously and therefore requires cloud manufacturing scheduling systems to be able to handle multiple tasks effectively. It is further complicated when multiple composite tasks are submitted to the system and to be addressed. A vast majority of existing studies have proposed various algorithms, including meta-heuristics, heuristics, and reinforcement learning algorithms, to address cloud manufacturing scheduling (CMfg-Sch) problems, but only a very small fraction of them deal with scheduling of multiple composition tasks with deep reinforcement learning. In this work, we leverage DRL coupled with sequence generation for addressing CMfg-Sch problems. Different from all existing works, we first propose two sequence generation algorithms for generating scheduling sequences of multiple composite tasks prior to scheduling. Coupled with this a Deep Q-Networks (DQN) and a Double DQN-based scheduling algorithms are proposed, respectively. Performance of the proposed algorithms is compared against seven baseline algorithms using makespan, cost, and reliability as evaluation metrics. Comparison indicates that sequence generation algorithm II (SGA-II) overall has a greater advantage over algorithm I (SGA-I), especially in terms of the makespan, and the Double DQN-based scheduling algorithm outperforms the DQN-based algorithm, which in turn performs better than other baseline algorithms.