When UAVs Meet Cognitive Radio: Offloading Traffic Under Uncertain Spectrum Environment via Deep Reinforcement Learning

Abstract

The emerging Internet of Things (IoT) paradigm makes our telecommunications networks increasingly congested. Unmanned aerial vehicles (UAVs) have been regarded as a promising solution to offload the overwhelming traffic. Considering the limited spectrums, cognitive radio can be embedded into UAVs to build backhaul links through harvesting idle spectrums. For the cognitive UAV (CUAV) assisted network, how much traffic can be actually offloaded depends on not only the traffic demand but also the spectrum environment. It is necessary to jointly consider both issues and co-design the trajectory and communications for the CUAV to make data collection and data transmission balanced to achieve high offloading efficiency, which, however, is non-trivial because of the heterogeneous and uncertain network environment. In this paper, aiming at maximizing the energy efficiency of the CUAV-assisted traffic offloading, we jointly design the Trajectory, Time allocation for data collection and data transmission, Band selection, and Transmission power control ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{T}^{\mathrm{ 3}}\text{B}$ </tex-math></inline-formula> ) considering the heterogeneous environment on traffic demand, energy replenishment, and spectrum availability. Considering the uncertain environmental information, we develop a model-free deep reinforcement learning (DRL) based solution to make the CUAV achieve the best decision autonomously. Simulation results have shown the effectiveness of the proposed DRL- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{T}^{\mathrm{ 3}}\text{B}$ </tex-math></inline-formula> strategy.