Smart Mode Selection Using Online Reinforcement Learning for VR Broadband Broadcasting in D2D Assisted 5G HetNets

Abstract

As an emerging broadband service pattern in the 5G era, VR broadcasting needs a considerable amount of bandwidth and strict quality of service (QoS) control. The traditional eMBMS or enTV transmission mode in HetNets consisting of macro cells and small cells cannot bring about a good trade-off between broadband performance and resource utilization for VR broadcasting service. D2D multicasting applied to VR broadcasting can improve the performance of edge users and resource utilization. Motivated by the rapid development of AI techniques, this paper proposes a novel hybrid transmission mode selection based on online reinforcement learning to address this problem. Each VR broadband user can be associated by one of the three modes: macrocell broadcasting, mmWave small cell unicasting and D2D multicasting. This paper first models this intelligent mode decision process as a problem to pursue the optimal system throughput. Then, an online machine learning-based method is proposed to solve this problem, which consists of a fast D2D clustering module based on unsupervised learning and a smart mode selection module based on reinforcement learning. The simulation results verify that the WoLF-PHC and Nash Q-learning perform better than other algorithms in large-scale scenarios and small-scale scenarios, respectively. The proposed intelligent transmission mode selection can also achieve larger VR throughput than traditional broadcasting strategies with a good balance between broadband performance and resource utilization.