Reinforcement Learning Aided UAV Base Station Location Optimization for Rate Maximization

Sudheesh Puthenveettil Gopi,Maurizio Magarini

doi:10.3390/electronics10232953

Sudheesh Puthenveettil Gopi, Maurizio Magarini

Open Access

https://doi.org/10.3390/electronics10232953

Copy DOI

Abstract

The application of unmanned aerial vehicles (UAV) as base station (BS) is gaining popularity. In this paper, we consider maximization of the overall data rate by intelligent deployment of UAV BS in the downlink of a cellular system. We investigate a reinforcement learning (RL)-aided approach to optimize the position of flying BSs mounted on board UAVs to support a macro BS (MBS). We propose an algorithm to avoid collision between multiple UAVs undergoing exploratory movements and to restrict UAV BSs movement within a predefined area. Q-learning technique is used to optimize UAV BS position, where the reward is equal to sum of user equipment (UE) data rates. We consider a framework where the UAV BSs carry out exploratory movements in the beginning and exploitary movements in later stages to maximize the overall data rate. Our results show that a cellular system with three UAV BSs and one MBS serving 72 UE reaches 69.2% of the best possible data rate, which is identified by brute force search. Finally, the RL algorithm is compared with a K-means algorithm to study the need of accurate UE locations. Our results show that the RL algorithm outperforms the K-means clustering algorithm when the measure of imperfection is higher. The proposed algorithm can be made use of by a practical MBS–UAV BSs–UEs system to provide protection to UAV BSs while maximizing data rate.

Highlights

The use of unmanned aerial vehicles (UAVs) in combination with terrestrial communication networks, in various capabilities, was initially considered for long-term evolution (LTE)
The main contribution of our work compared with existing work is as follows: (i) the macro BS (MBS) is considered only as backhaul in [25], whereas, in this work, the MBS serves users, in addition to backhaul; (ii) we make use of a scheme proposed in [29], where the UAV base station (BS) explore more in the initial stages and exploit in later stages, with an aim to maximize the overall data rate; (iii) in addition to implementation of reinforcement learning (RL) assisted UAV BS positioning, we propose a scheme to avoid collision among multiple UAV BSs; and (iv) considering the exploratory nature of UAV BS, we propose a method to avoid UAV BSs from moving out of the desired service area
In the first step corresponding to episode 0, UAV BSs are placed at origin

Summary

Introduction

The use of unmanned aerial vehicles (UAVs) in combination with terrestrial communication networks, in various capabilities, was initially considered for long-term evolution (LTE). In LTE, UAVs were considered both as flying user equipment (UE), widely known as cellular connected UAVs, and as flying base station (BS). The cellular connected UAVs were used extensively for surveying, acquiring sensor data etc., whereas the UAV BSs are proposed to play a major role especially during natural calamities and similar situations where ground based structure might be absent [1]. Apart from providing the necessary communication service during a natural calamity, a UAV can be used to improve the performance of an existing network. The UAV BSs are still costly, and researchers from all over the world are working on prototypes of various capacities. In 2020, Verizon experimented with a 300 pound prototype [2], clearly concluding that a cost effective UAV

Results

Discussion

Conclusion