Deep-Reinforcement-Learning-Based Drone Base Station Deployment for Wireless Communication Services

Abstract

Over the last few years, drone base station (DBS) technology has been recognized as a promising solution to the problem of network design for wireless communication systems, due to its highly flexible deployment and dynamic mobility features. This article focuses on the 3-D mobility control of the DBS to boost transmission coverage and network connectivity. We propose a dynamic and scalable control strategy for drone mobility using deep reinforcement learning (DRL). The design goal is to maximize communication coverage and network connectivity for multiple real-time users over a time horizon. The proposed method functions according to the received signals of mobile users, without the information of user locations. It is divided into two hierarchical stages. First, a time-series convolutional neural network (CNN)-based link quality estimation model is used to determine the link quality at each timeslot. Second, a deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning algorithm is applied to control the movement of the DBS in hotspot areas to meet user requirements. Simulation results show that the proposed method achieves significant network performance in terms of both communication coverage and network throughput in a dynamic environment, compared with the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning algorithm.