Multi-objective deep reinforcement learning for computation offloading in UAV-assisted multi-access edge computing

Abstract

Unmanned aerial vehicle-assisted multi-access edge computing (UAV-MEC) plays an important role in some complex environments such as mountainous and disaster areas. Computation offloading problem (COP) is one of the key issues of UAV-MEC, which mainly aims to minimize the conflict goals between energy consumption and delay. Due to the time-varying and uncertain nature of the UAV-MEC system, deep reinforcement learning is an effective method for solving the COP. Different from the existing works, in this paper, the COP in UAV-MEC system is modeled as a multi-objective Markov decision process, and a multi-objective deep reinforcement learning method is proposed to solve it. In the proposed algorithm, the scalar reward of reinforcement learning is expanded into a vector reward, and the weights are dynamically adjusted to meet different user preferences. The most important preferences are selected by non-dominated sorting, which can better maintain the previously learned strategy. In addition, the Q network structure combines Double Deep Q Network (Double DQN) with Dueling Deep Q Network (Dueling DQN) to improve the optimization efficiency. Simulation results show that the algorithm achieves a good balance between energy consumption and delay, and can obtain a better computation offloading scheme.