Federated Deep Reinforcement Learning for Online Task Offloading and Resource Allocation in WPC-MEC Networks

Lianqi Zang,Xin Zhang,Boren Guo

doi:10.1109/access.2022.3144415

Lianqi Zang, Xin Zhang + Show 1 more

Open Access

https://doi.org/10.1109/access.2022.3144415

Copy DOI

Abstract

Mobile edge computing (MEC) is considered a more effective technological solution for developing the Internet of Things (IoT) by providing cloud-like capabilities for mobile users. This article combines wireless powered communication (WPC) technology with an MEC network, where a base station (BS) can transfer wireless energy to edge users (EUs) and execute computation-intensive tasks through task offloading. Traditional numerical optimization methods are time-consuming approaches for solving this problem in time-varying wireless channels, and centralized deep reinforcement learning (DRL) is not stable in large-scale dynamic IoT networks. Therefore, we propose a federated DRL-based online task offloading and resource allocation (FDOR) algorithm. In this algorithm, DRL is executed in EUs, and federated learning (FL) uses the distributed architecture of MEC to aggregate and update the parameters. To further solve the problem of the non-IID data of mobile EUs, we devise an adaptive method that automatically adjusts the FDOR algorithm’s learning rate. Simulation results demonstrate that the proposed FDOR algorithm is superior to the traditional numerical optimization method and the existing DRL algorithm in four aspects: convergence speed, execution delay, overall calculation rate and stability in large-scale and dynamic IoT.

Highlights

The internet of things (IoT) technology has entered into the stage with the comprehensive combination of artificial intelligence (AI) and 5G network technology [1]
Since WPT is greatly affected by the transmission distance, and the calculation cycle of tasks has a great influence on the execution delay of the mobile edge computing (MEC) server, we evaluated the effects of the communication distance, task calculation cycle on the calculation rate
In this paper, we propose an online offloading algorithm FDOR based on the combination of deep reinforcement learning (DRL) and federated learning (FL)

Summary

Introduction

The internet of things (IoT) technology has entered into the stage with the comprehensive combination of artificial intelligence (AI) and 5G network technology [1]. (2) Because a large number of IoT devices are located in coverage areas, deployment costs will rise substantially by replacing the battery manually or utilizing wired charging. Evolution of mobile edge computing (MEC) technology enables EUs to offload computing-intensive and delaysensitive tasks to MEC servers, effectively completing more complicated work [3]. The combination of MEC and WPC technology has solved the limitations of IoT devices in terms of battery charging and computing power, providing a better environment for IoT development [4]. The advantage of the WPC-MEC network is the deployment of energy stations near EUs. The energy station provides power for EUs in real time through WPC, and EUs apply the collected energy to transmit their computing tasks to the MEC server.

Objectives

Results

Conclusion