ESN Reinforcement Learning for Spectrum and Flight Control in THz-Enabled Drone Networks

Abstract

Terahertz (THz)-band communications have been envisioned as a key technology to support ultra-high-data-rate applications in 5G-beyond (or 6G) wireless networks. Compared to the microwave and mmWave bands, the main challenges with the THz band are in its i) large path loss hence limited network coverage and ii) visible-light-like propagation characteristics hence poor support of mobility in blockage-rich environments. This paper studies quantitatively the applicability of THz-band communications in blockage-rich mobile environments, focusing on a new network scenario called FlyTera. In FlyTera, a set of hotspots mounted on flying drones collaboratively provide data streaming services to ground users, in the microwave, mmWave and THz bands. We first provide a mathematical formulation of the FlyTera control problem, where the objective is to maximize the network spectral efficiency by jointly controlling the flight of the drone hotspots, their association to the ground users, and the spectrum bands used by the users. To solve the resulting problem, which is shown to be a mixed integer nonlinear nonconvex programming (MINLP) problem, we design distributed solution algorithms based on a combination of echo state learning and reinforcement learning. An extensive simulation campaign is then conducted with SimBAG, a newly developed Simulator of Broadband Aerial-Ground wireless networks. It is shown that no single spectrum band can meet the requirements of high data rate and wide coverage simultaneously. Moreover, from the network-level point of view, THz-band communications can significantly benefit from the mobility of the flying drones, and on average 4 - 6 times higher (rather than lower) throughput can be achieved in mobile than in static environments.