FlyReflect: Joint Flying IRS Trajectory and Phase Shift Design Using Deep Reinforcement Learning

Abstract

Aerial access infrastructures have been considered a compulsory component of the sixth-generation (6G) networks, where airborne vehicles play the role of mobile access points to service ground users (GUs) from the sky. In this scenario, intelligent reflecting surface (IRS) is one of the promising technologies associated with airborne vehicles for coverage extensions and throughput improvements, a.k.a., flying IRS (F-IRS). This study considers a multiuser multiple-input single-output (MISO) F-IRS system, where the F-IRS reflects downlink signals from ground base stations (BSs) to users located at underserved areas where direct communications are unavailable. To achieve the system sum-rate maximization, we proposed a deep reinforcement learning (DRL) algorithm named <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FlyReflect</monospace> to jointly optimize the flying trajectory and IRS phase shift matrix. First, end-to-end communications from a BS to its GUs via the F-IRS are analyzed to identify environmental and operational factors that impact achievable system sum rate. Subsequently, the system is transformed into a DRL model, which is resolvable by the deep deterministic policy gradient (DDPG) algorithm. To improve the action decision accuracy of the DDPG algorithm, we proposed a mapping function to guarantee that all constraints are satisfied regardless of noise additions in the exploration process. Simulation results showed that our proposed algorithm outperforms state-of-the-art algorithms in multiple scenarios.