A Deep Reinforcement Learning Approach for Shared Caching

Abstract

A client-server network in which multiple clients are connected to a single server possessing files/data through a shared error free link is considered. Each client is associated with a cache memory and demands a file from the server. The server loads the cache memory with a portion of files during off-peak hours to reduce the delivery rate during peak hours. A decentralized placement approach which is more practical for large networks is considered for filling the cache contents. In this paper, the shared caching problem in which each cache can be accessed by multiple clients is considered. A Deep Reinforcement Learning (DRL) based framework is proposed for optimizing the delivery rate of the requested contents by the users. The system is strategically modelled as a Markov decision process, to deploy our DRL agent and enable it to learn how to make decisions. The DRL agent learns to multicast coded bits from the file library of the server in such a way that the user requests are met with minimum transmissions of these coded bits. It is shown that the proposed DRL based agent outperforms the existing decentralized algorithms for the shared caching problem in terms of normalized delivery rate. For the conventional caching problem which is a special case of the shared caching problem, simulation results show that the proposed DRL agent outperforms the existing algorithms.