Real-Time Energy Harvesting Aided Scheduling in UAV-Assisted D2D Networks Relying on Deep Reinforcement Learning

Khoi Khac Nguyen,Minh-Tuan Le,Long D Nguyen,Lajos Hanzo,Ngo Anh Vien,Trung Q Duong

doi:10.1109/access.2020.3046499

Abstract

Unmanned aerial vehicle (UAV)-assisted device-to-device (D2D) communications can be deployed flexibly thanks to UAVs' agility. By exploiting the direct D2D interaction supported by UAVs, both the user experience and network performance can be substantially enhanced at public events. However, the continuous moving of D2D users, limited energy and flying time of UAVs are impediments to their applications in real-time. To tackle this issue, we propose a novel model based on deep reinforcement learning in order to find the optimal solution for the energy-harvesting time scheduling in UAV-assisted D2D communications. To make the system model more realistic, we assume that the UAV flies around a central point, the D2D users move continuously with random walk model and the channel state information encountered during each time slot is randomly time-variant. Our numerical results demonstrate that the proposed schemes outperform the existing solutions. The associated energy efficiency game can be solved in less than one millisecond by an off-the-shelf processor using trained neural networks. Hence our deep reinforcement learning techniques are capable of solving real-time resource allocation problems in UAV-assisted wireless networks.

Highlights

Unmanned aerial vehicles (UAVs) have various wireless applications ranging from public safety, environmental monitoring, and enhanced network connectivity as a benefit of their nimble mobility features
ENERGY HARVESTING TIME SCHEDULING IN UAV-POWERED D2D COMMUNICATIONS: A DEEP DETERMINISTIC POLICY GRADIENT APPROACH we propose a deep deterministic policy gradient algorithm (DDPG) [33] for energy harvesting time scheduling in UAV-powered D2D communications
EFFICIENT LEARNING WITH PROXIMAL POLICY OPTIMISATION ALGORITHMS TO SOLVE THE ENERGY HARVESTING TIME SCHEDULING PROBLEM IN D2D COMMUNICATIONS ASSISTED BY UAV we propose a novel model based on the policy optimisation (PPO) algorithm relying on an efficient sampling technique for solving the energy harvesting time scheduling game in UAV-assisted D2D communications

Summary

Introduction

Unmanned aerial vehicles (UAVs) have various wireless applications ranging from public safety, environmental monitoring, and enhanced network connectivity as a benefit of their nimble mobility features. UAVs are capable of assisting wireless networks in providing ubiquitous coverage, robust handovers, and flawless real-time multi-media communica-. The associate editor coordinating the review of this manuscript and approving it for publication was Guangjie Han. tions. The performance of UAV-assisted networks is limited by the UAVs’ energy-storage and the resultant flying time. Recent research has tackled some of the challenges in UAV-supported wireless communications [1]–[9]. Most techniques rely on unrealistic simplifications and focus predominantly on data transmission. It is crucial to find solutions to the associated problems in realistic dynamic environments, as detailed below

Objectives

Results

Conclusion