Dynamic Spectrum Sharing in Cellular Based Urban Air Mobility via Deep Reinforcement Learning

Abstract

With increasing mobility demands in metropolitan areas, the emerging concept of Urban Air Mobility (UAM) opens a new urban air transportation paradigm, where one big challenge is to ensure reliable two-way communications between aerial vehicles and their ground control stations for safe operations. The concept of cellular-based UAM (cUAM) provides a promising solution where aerial vehicles are regarded as new aerial users, sharing the cellular spectrum with existing terrestrial users. However, with new characteristics and demands of aerial users, the severity of spectrum scarcity becomes more prominent and new spectrum management solutions are needed. In this paper, we consider a typical cUAM scenario where multiple aerial vehicles transport passengers/cargo along their pre-defined paths, with the coexistence of terrestrial users. We assume the minimum communication Quality of Service (QoS) must be achieved for all users at all times. Our objective is to simultaneously minimize aerial users' mission completion time and maximize terrestrial users' achievable data rate by jointly optimizing the spectrum allocation for all users and the moving velocities of aerial users. We formulate the optimization problem as a multi-stage Markov Decision Process and propose a multi-agent deep reinforcement learning-based algorithm. We also propose a heuristic greedy algorithm and an orthogonal multiple access algorithm as baseline solutions. Simulation results show that our learning-based solution outperforms the baseline solutions under different network configurations.