Proactive Caching for Vehicular Multi-View 3D Video Streaming via Deep Reinforcement Learning

Abstract

This paper investigates the problem of proactive caching for multi-view 3D videos in the fifth generation (5G) networks. We establish a mathematical model for this problem, and point out that it is difficult to solve the problem with traditional dynamic programming, then we propose a deep reinforcement learning approach to solve it. First, we model the proactive caching system for multi-view 3D videos as a Markov decision process jointing views selection and local memory allocation. Then, we present an actor-critic, model-free algorithm based on the deep deterministic policy gradient to find effective proactive caching policy. Since the action space is affected by the system state, we embed dynamic k-Nearest Neighbor algorithm into actor-critic algorithm to implement the deep reinforcement learning algorithm working in an action space of variable size. Finally, the numerical results are given to demonstrate that the proposed solution can effectively maintain high-quality user experience for high-mobility 5G users moving among small cells. We also investigate the impact of configuration of critical parameters on the performance of the algorithm.