Computation Offloading in Energy Harvesting Systems via Continuous Deep Reinforcement Learning

Abstract

As a promising technology to improve the computation experience for mobile devices, mobile edge computing (MEC) is becoming an emerging paradigm to meet the tremendous increasing computation demands. In this paper, a mobile edge computing system consisting of multiple mobile devices with energy harvesting and an edge server is considered. Specifically, multiple devices decide the offloading ratio and local computation capacity, which are both in continuous values. Each device equips a task load queue and energy harvesting, which increases the system dynamics and leads to the time-dependence of the optimal offloading decision. In order to minimize the sum cost of the execution time and energy consumption in the long-term, we develop a continuous control based deep reinforcement learning algorithm for computation offloading. Utilizing the actor-critic learning approach, we propose a centralized learning policy for each device. By incorporating the states of other devices with centralized learning, the proposed method learns to coordinate among all devices. Simulation results validate the effectiveness of our proposed algorithm, which demonstrates superior generalization ability and achieves a better performance compared with discrete decision based deep reinforcement learning methods.