Model-Free Ultra Reliable Low Latency Communication (URLLC): A Deep Reinforcement Learning Framework

Abstract

In this paper, a novel deep reinforcement learning (deep-RL) framework is proposed to provide model-free ultra reliable low latency communication (URLLC) in the downlink of an orthogonal frequency division multiple access (OFDMA) system. The proposed deep-RL framework can guarantee high end-to-end reliability and low end-to-end latency, under data rate constraints, for each user in the cellular system without any models of or assumptions on the users' traffic. Using the proposed model-free approach, the users' traffic is predicted by the deep-RL framework and subsequently used in the resource allocation, irrespective of the actual underlying model. The problem is posed as a power minimization problem under reliability, latency, and rate constraints. To solve this problem using deep-RL, first, the rate of each user is determined. Then, these rates are mapped to the resource block and power allocation vectors of the studied OFDMA system. Finally, the end-to-end reliability and latency of each user are used as a feedback to the deep-RL framework. It is shown that at the fixed-point of the deep-RL algorithm, the reliability and latency of the users are guaranteed. Simulation results show how the proposed approach can achieve any feasible point in the rate-reliability-latency region, depending on the network and service requirements. For example, for a 7 Mbps rate guarantee, the results show that the proposed algorithm can provide ultra-reliable low latency communication with a delay of 8 milliseconds and a reliability of 98%.