Online computation offloading with double reinforcement learning algorithm in mobile edge computing

Abstract

Smart mobile devices have recently emerged as a promising computing platform for computation tasks. However, the task performance is restricted by the computing power and battery capacity of mobile devices. Mobile edge computing, an extension of cloud computing, solves this problem well by providing computational support to mobile devices. In this paper, we discuss a mobile edge computing system with a server and multiple mobile devices that need to perform computation tasks with priorities. The limited resources of the mobile edge computing server and mobile device make it challenging to develop an offloading strategy to minimize both delay and energy consumption in the long term. To this end, an online algorithm is proposed, namely, the double reinforcement learning computation offloading (DRLCO) algorithm, which jointly decides the offloading decision, the CPU frequency, and transmit power for computation offloading. Concretely, we first formulate the power scheduling problem for mobile users to minimize energy consumption. Inspired by reinforcement learning, we solve the problem by presenting a power scheduling algorithm based on the deep deterministic policy gradient (DDPG). Then, we model the task offloading problem to minimize the delay of tasks and propose a double Deep Q-networks (DQN) based algorithm. In the decision-making process, we fully consider the influence of task queue information, channel state information, and task information. Moreover, we propose an adaptive prioritized experience replay algorithm to improve the model training efficiency. We conduct extensive simulations to verify the effectiveness of the scheme, and the simulation results show that compared with the conventional schemes, our method reduces the delay by 48% and the energy consumption by 53%.