A Decoupled Access Scheme With Reinforcement Learning Power Control for Cellular-Enabled UAVs

Abstract

This article proposes a downlink/uplink decoupled (DUDe) access scheme for cellular-enabled unmanned aerial vehicle (UAV) communication systems. To minimize interference, the proposed scheme separates the control and data links of UAVs, as well as the uplinks (ULs) and downlinks (DLs) of ground users (GUEs), onto different serving base stations and operating frequencies. Since power availability is a major constraint in UAV communications, two power allocation schemes based on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning (QL) and deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning (DQL) are proposed to optimize the communication energy efficiency (EE) of this DUDe network. To quantify the improvements achieved, the proposed schemes are compared with the fractional power control (FPC) scheme used in 4G and 5G networks, as well as a convex optimization-based optimal power allocation scheme. The results demonstrate that the proposed DUDe scheme can achieve up to several times higher sum rates and EE in the UL direction than its coupled counterparts. Moreover, it is shown that the EE performance of the QL and DQL power allocation schemes approach the optimal performance and surpass the conventional FPC scheme by 80%–100% in the UHF band, and by 160%–170% in the mmWave band.