Cooperative Caching and Transmission in CoMP-Integrated Cellular Networks Using Reinforcement Learning

Abstract

Mobile network caching (MNC) and Coordinated MultiPoint (CoMP) joint transmission (JT) techniques are expected to play a significant role in future networks. Yet, when considering caching in a CoMP-integrated network where both single transmission (ST) and JT are permitted, some challenges need to be tackled. In this paper, we propose a cooperative online caching strategy to realize autonomous transmission-aware content caching and updating for video content delivery. Different from existing works, this paper considers practical time-varying user request patterns, and exploits storage-level and transmission-level cooperation to jointly optimize content caching under dynamic user demands for JT and ST. The former helps reduce caching redundancy and the latter helps BSs create CoMP-JT opportunities to improve cell-edge throughput. We formulate the content caching problem to be a Markov decision process (MDP), in which the system reward is defined as the delay reduction after performing a caching action. The goal is to maximize the system reward. Then, we develop a reinforcement learning (RL)-based online learning algorithm to search the optimal caching policy. To overcome the “curse of dimensionality,” the designed RL-based algorithm is extended with linear approximation. The system performance in terms of delay and cache hit rate is evaluated in the simulations, and the results demonstrate the effectiveness of the proposed strategy.