Maximizing profitability through cloud-enabled Reinforcement Learning for UAV coverage in real-time e-business applications

Abstract

Real-time e-business applications are vital for operational efficiency, but connectivity challenges persist, particularly in remote or crowded areas. Drone Base Station (DBS) architecture, proposed for Beyond fifth Generation (B5G) and Sixth Generation (6G) multi-cell networks, offers on-demand hotspot coverage, addressing connectivity gaps in remote or crowded environments. DBSs provide a promising solution to meet the demanding requirements of high data rates, real-time responsiveness, low latency, and extended network coverage, particularly for real-time e-business applications. A critical challenge in this context involves efficiently allocating the needed number of DBSs to the different hotspot service areas, referred to as cells, to optimize the operator’s total profit under unpredictable user demands, varying area-specific service costs, and price dependence real-time e-service. The objective is to achieve the highest total revenue while minimizing the cost (cost savings) throughout the multi-cell system. This challenge is formulated as a profit-maximization discount return problem, integrating the coverage constraint, the variable cell-dependent operational cost, the e-service-based price and the uncertain demands of users across cells. Traditional optimization methods fail due to environmental uncertainty, which leads to the need to reformulate the problem as a Markov Decision Problem (MDP). We introduce a cloud-based Reinforcement Learning (RL) algorithm for DBS dispatch to address the MDP formulation. This algorithm dynamically adjusts to uncertain per-cell user distributions, considering variable operating costs and service-dependent prices across cells. Through extensive evaluation, the RL-based dispatch approach is compared with reference drone dispatch algorithms, demonstrating superior performance in maximizing operator profit through cost savings by optimizing DBS dispatch decisions based on learned user behaviors, variable operational costs, and e-service types.