Deep Reinforcement Learning-Aided Optimization of Multi-Interface Allocation for Short-Packet Communications

Abstract

The severe spectrum scarcity and the stringent requirements of Beyond 5G applications call for an integrated use of low frequency Sub-6 GHz and high frequency millimeter Wave bands. Focusing on future Internet of Things (IoT) Short-Packet Communications (SPC), this paper investigates the optimized usage of such diverse wireless interfaces. We propose an unifying framework devoted to SPC that jointly optimizes the user partitioning over each band, and the radio resource scheduling within each band. Leveraging Deep Reinforcement Learning (DRL) tools, the proposed method enables to better tackle the challenges imposed by dynamically varying mobile environments such as the Line-of-Sight situations of each link, and the heterogeneity of individual Quality of Service (QoS) requirements, such as rate, delay and reliability. Regarding the DRL-based user partitioning to each band, we have investigated three different types of partitioning actions to obtain a high network performance as well as a rapid convergence. Regarding the proposed sub-schedulers within each band, we designed two optimization methods, i.e., one that leverages Difference of Convex Programming (DCP) technique, and the second that accelerates convergence to a local optimum. Numerical evaluations show that the proposed methods outperform conventional approaches in terms of sum-rate and QoS outage probabilities.