Integrated Resource Scheduling for User Experience Enhancement: A Heuristically Accelerated DRL

Abstract

The expected explosive traffic has forced the fifth-generation (5G) mobile communication system to be ultra-dense networks (UDNs). Driven by diverse applications, flexible resource scheduling encounters novel challenges. Resource scheduling issue is a large-scale optimization problem with high complexity and computational cost, which always leads to a nearly-optimal solution instead of a truly-optimal one. Moreover, resource scheduling is specific for partial traffic types, which cannot well support diverse quality-of-service (QoS) demands in terms of reliability, throughput, and delay. In addition, current resource scheduling problem is separately solved as user scheduling and resource allocation, by which the submodule optimization may not be the global one. In this paper, we propose an integrated scheduling framework with both user scheduling and resource allocation into consideration, which intelligently implement automatic radio resource management. Furthermore, we design the optimal resource scheduling strategy to maximize the user satisfaction based on deep reinforcement learning (DRL). We also add the experience replay, main network (MainNet) and target network (TargetNet), and heuristic mechanism into conventional DRL framework for the performance enhancement and convergence acceleration. Simulation results confirm the effectiveness of the proposed algorithm in user experience improvement.