Cloud–edge collaboration task scheduling in cloud manufacturing: An attention-based deep reinforcement learning approach

Abstract

Cloud Manufacturing (CMfg), as a service-oriented manufacturing mode, aims to provide consumers on-demand manufacturing services. The CMfg platform requires task scheduling technology to schedule manufacturing tasks efficiently, and improve resource utilization and customer satisfaction. Existing scheduling models for manufacturing tasks mainly consider maximizing the quality of service for customers but ignore the actual production execution, which will lead to low-quality execution or delayed delivery. To maximize customer satisfaction and balance production, this article studies a cloud–edge collaboration manufacturing task scheduling in CMfg (CETS). CETS refines manufacturing services deployed in the cloud to the factory process level, and schedules tasks according to the real-time production information on the edge side and manufacturing service information on the cloud side. Considering the dynamics of CETS and the complexity of state information in CETS, an attention-based deep reinforcement learning (DRL) algorithm is proposed to solve CETS. First, the CETS is mathematically represented and built as a partially observable Markov decision process. Second, on-policy maximum a posteriori policy optimization (V-MPO) with gated transformer-XL (GTrXL) named AV-MPO is developed. The effectiveness, training stability, generalizability, scalability, and robustness of AV-MPO are investigated. Rule-based algorithms and some state of art DRL algorithms, such as proximal policy optimization (PPO), soft actor-critic (SAC), and dueling deep q network (Dueling DQN), are compared with AV-MPO. The experimental results validate that AV-MPO can deal with the CETS problem more effectively.