Deep Q-Network based resource allocation for UAV-assisted Ultra-Dense Networks

Abstract

With the rapid development of the fifth-generation (5G) wireless communications, the number of users is increasing dramatically and Ultra-Dense Networks (UDN) are becoming more important for supporting numerous users and emerging mission-critical applications. In order to conquer the communication restrictions caused by natural disasters, an emergency communication system using Unmanned Aerial Vehicles (UAV) as a flying base station (BS) to assist UDN is proposed. By virtue of the resource allocation scheme of UAV-assisted UDN systems, communication resources can be reasonably and effectively allocated to improve the quality of user experience. Firstly, aiming to maximize the system energy efficiency (EE), a UDN system model including the BS selection is constructed. Secondly, Markov Decision Process (MDP) theory is applied to transform the system model into a stochastic optimization problem. Finally, by using deep reinforcement learning (DRL) technique, we propose a Deep Q-Network (DQN) based resource allocation scheme to maximize the system energy efficiency. Simulation results exhibit that the proposed DQN-based resource allocation scheme can significantly improve the system EE compared with the legacy Q-Learning, random and maximum resource allocation algorithms.