Q-Learning-Based Power Allocation for Secure Wireless Communication in UAV-Aided Relay Network

Abstract

Unmanned aerial vehicle (UAV)-aided wireless relay networks are at risk of eavesdropping activities due to their open nature. In this paper, we study the security of a UAV-aided selective relaying wireless network in which $N$ UAVs are employed as decode-and-forward (DF) relays linking a ground base station (BS) with $L$ legitimate users on the ground in the presence of a passive eavesdropper ( $Eave$ ). Direct links between the ground BS and both the ground users and the eavesdropper are assumed to be blocked. The ground-to-air and air-to-ground channels are assumed to follow Rician fading model with opportunistic scheduling scheme for UAVs and users selection. In order to secure data transmissions against such an interception action, the UAV of the worst UAV-selected user link transmits a jamming artificial noise (AN) signal to degrade $Eave$ ability in decoding the confidential information successfully. The transmission outage probability, intercept probability, and hybrid outage probability are derived and analyzed. Due to the heavy computation burden raised by increasing the number of UAVs and users as well as the difficulty in estimating the instantaneous channel state information (CSI), existing traditional optimization methods are not highly efficient in solving the considered power allocation problem. Therefore, we propose a dynamic power control scheme based on Q-learning algorithm combined with statistical CSI where the hybrid outage probability is minimized. Simulation results show that the proposed algorithm efficiently reduces the hybrid outage probability with a noticeable reduction in the computational time.