Multiagent Multi-Armed Bandit Schemes for Gateway Selection in UAV Networks

Abstract

Lately, unmanned aerial vehicles (UAVs) communications acquired great attention because of its weighty new applications, particularly in rescue services. In such a case, access and gateway UAVS are spread to cover and fully support communications over disaster areas where the ground network is malfunctioned or wholly damaged. Each access UAV collects essential information from its assigned area, then flies and transfers it to the nearby gateway UAVs that deliver this collected information to the closest operating ground network. Meanwhile, collisions may occur as two or more access UAVs might target the same gateway UAV. This paper leverages and modifies two multi-armed bandit (MAB) based algorithms, namely, Kullback Leibler upper confidence bound (KLUCB) and minimax optimal stochastic strategy (MOSS) to formulate the gateway UAV selection issue. The issue is modeled as a budget-constrained multiagent MAB (MA-MAB) that maximizes data rates while considering access UAVs’ flight battery consumption. Hence, MA battery aware KLUCB (MABA-KLUCB) and battery aware MOSS (MA-BA-MOSS) algorithms are proposed for efficient gateway UAV selection. The proposed MAB algorithms maximize the UAV network’s total sum rate over the conventional selection techniques with assuring good convergence performance.