Soft Actor-Critic-Based DAG Tasks Offloading in Multi-access Edge Computing with Inter-user Cooperation

Abstract

AbstractMulti-access edge computing (MEC) enables mobile applications, which consists of multiple dependent subtasks, to be executed in full parallelism between edge servers and mobile devices. The idea is to offload some of the subtasks to edge servers while utilizing the directed acyclic graphs (DAGs) based subtask dependency. In some cases involving multiple users running the same application, users can cooperate to improve the application’s performance (e.g., object detection accuracy in connected and autonomous vehicles) by sharing the intermediate results of the subtasks, which is referred to as cooperation gain. However, inter-user cooperation also introduces additional dependencies between the subtasks of different users, which inevitably increases the execution delay of the application with DAG tasks offloading. Therefore, how to jointly optimize execution delay and cooperation gain remains an open question. In this paper, we present an approach for DAG tasks offloading in MEC with inter-user cooperation to optimize the execution delay of the application and the cooperation gain. First, we introduce the concept of application utility, which incorporates both delay and cooperation gain. We formulate the DAG tasks offloading problem with inter-user cooperation as a Markov decision process (MDP) to maximize the total application utility. Next, we propose a quantized soft actor-critic (QSAC) algorithm to solve the formulated problem. Specifically, QSAC generates multiple potential solutions according to the order-preserving quantizing method, increasing the exploration efficiency to avoid falling into local optimal while maintaining the overall stability. Finally, simulation results validate that the proposed algorithm significantly improves the total application utility compared with the other benchmark algorithms.KeywordsMulti-access edge computing (MEC)Tasks offloadingDirected acyclic graphs (DAGs)Inter-user cooperationMarkov decision process (MDP)Soft actor-critic (SAC)