Deep Reinforcement Learning Based Cooperative Partial Task Offloading and Resource Allocation for IIoT Applications

Abstract

Compared with traditional cloud computing schemes, edge computing provides an effective solution towards meeting real-time, security and reliability demands of compute-intensive and delay-sensitive Industrial Internet of Things (IIoT) applications. However, many existing studies only optimize end-edge-cloud cooperative task offloading, and neglect the optimization of the communication and computation resource allocation. In this paper, the cooperative partial task offloading and resource allocation (CPTORA) framework is designed, which jointly considers cooperation among various IIoT devices, local edge computing servers (ECSs), non-local ECSs, and cloud computing servers. Then, considering the complex dynamics and unpredictability in IIoT environments, the joint optimization problem is modeled as a constrained Markov decision process. Furthermore, we propose an improved soft actor-critic-based CPTORA (ISAC-CPTORA) algorithm, to make task offloading and resource allocation decisions for each IIoT device. This algorithm innovatively introduces the idea of distributional reinforcement learning to the soft actor-critic, which can effectively reduce Q-value overestimations or underestimations. Meanwhile, this algorithm employs the prioritized experience replay to enhance its learning efficiency. Extensive laboratory experiments indicate that our CPTORA framework and ISAC-CPTORA algorithm efficiently decrease the total system costs, in contrast to various baseline frameworks and algorithms.