Deployment and Movement for Multiple Aerial Base Stations by Reinforcement Learning

Abstract

A novel framework for Quality of experience (QoE)-driven deployment and movement of multiple unmanned aerial vehicles (UAVs) is proposed. The problem of joint non-concave 3D deployment and dynamic movement for maximizing the sum mean opinion score (MOS) of users is formulated, which is proved to be NP-hard. In an effort to solve this problem, we proposed a three-step approach to obtain 3D deployment and dynamic movement of multiple UAVs. More specifically, in the first step, GAK-means algorithm is invoked to obtain the cell partitioning of ground users. Secondly, Q-learning based deployment algorithm is proposed, in which each UAV is considered as an agent, making its own decision to obtain 3D position. In contrast to conventional genetic algorithm based learning algorithms, the proposed algorithm is capable of training the policy of making decision offline. Thirdly, Q-learning algorithm is invoked when the ground users roam. Unlike the other trajectory obtaining algorithms, the proposed approach enables each UAV learn its movement gradually through trials and errors, and updates the direction selection strategy until it reaches convergence. Numerical results reveal that the proposed 3D deployment scheme outperforms K-means algorithm and IGK algorithm with low complexity. Additionally, with the aid of proposed approach, 3D real-time dynamic movement of UAVs is obtained.