Unmanned-Aerial-Vehicle-Assisted Computation Offloading for Mobile Edge Computing Based on Deep Reinforcement Learning

Hui Wang,Hongchang Ke,Weijia Sun

doi:10.1109/access.2020.3028553

Abstract

Users in heterogeneous wireless networks may generate massive amounts of data that are delay-sensitive or require computation-intensive processing. Owing to computation ability and battery capacity limitations, wireless users (WUs) cannot easily process such data in a timely manner, and mobile edge computing (MEC) is increasingly being used to resolve this issue. Specifically, data generated by WUs can be offloaded to the MEC server for processing, which has greater computing power than WUs. However, as the location of MEC servers is fixed, unmanned aerial vehicles (UAVs) have been considered a promising solution in heterogeneous wireless networks. In this study, we design an UAV-assisted computation offloading scheme in an MEC framework with renewable power supply. The proposed model considers the instability of energy arrival, stochastic computation tasks generated by WUs, and a time-varying channel state. Owing to the complexity of the state, it is difficult to use traditional Markov decision process (MDP) with complete prior knowledge for offloading optimization. Accordingly, we propose UAV-assisted computation offloading for MEC based on deep reinforcement learning (UACODRL) to minimize the total cost, which is the weighted sum of the delay, energy consumption, and bandwidth cost. We first use the K-Means algorithm for classification to reduce the dimension of the action space. Subsequently, we use UACODRL to find the near-optimal offloading scheme to minimize the total cost. Simulations demonstrate that UACODRL converges satisfactorily and performs better than four baseline schemes with different parameter configurations.

Highlights

An increasing number of computation tasks related to, for instance, virtual and augmented reality, or online gaming, are generated in wireless networks, and some of these tasks are delay-sensitive or computation-intensive [1], [2]
In light of recent research on deep reinforcement learning (DRL), we propose unmanned aerial vehicles (UAVs)-assisted computation offloading for mobile edge computing (MEC) based on DRL (UACODRL)
The Follow-Me Cloud (FMC) controller in the proposed UAV-assisted MEC framework can obtain all global information of wireless users (WUs), UAVs and MEC server which can guide the agent to execute computation offloading to UAVs or MEC server for minimizing the global cost

Summary

Introduction

An increasing number of computation tasks related to, for instance, virtual and augmented reality, or online gaming, are generated in wireless networks, and some of these tasks are delay-sensitive or computation-intensive [1], [2]. Shahzad et al proposes an efficient dynamic programming scheme using the Hamming distance termination algorithm to optimize computation offloading. In this scheme, multiple users offload tasks to a cloud server according to the state of the wireless network bandwidth [9]. Goudarzi et al employs an optimized version of particle swarm optimization to obtain a near-optimal offloading solution and reduce the total energy consumption [10]. In these methods, it is difficult to handle resource scheduling and backhaul delay issues

Objectives

Methods

Results

Conclusion