Caching popularity-unknown contents for massive communication channels via deep Q-learning

Abstract

In order to reduce the backhaul link pressure of wireless networks, edge caching technology has been regarded as a promising solution. However, with massive and dynamical communication connections, it is challenging to provide analytical caching solution to achieve the best performance, particularly when the requested contents are changing and their popularities are unknown. In this paper, we propose a deep Q-learning (DQN) method to address the issue of caching placement. Considering a content caching network containing multiple cooperating SBSs with unknown content popularity, we need to determine which content to cache and where to cache. Therefore, the learning network has to be designed for dual aims, one of which is to estimate the content popularities while the other is to assign contents to the proper channels. An elaborate DQN is proposed to make decisions to cache contents with limited storage space of base-station by considering channel conditions. Specifically, the content requests of users are first collected as one of the training samples of the learning network. Second, the channel state information for the massive links are estimated as the other training samples. Then, we train the network based on the proposed method thereby improving spectral efficiency of the entire system and reducing bit-error rate. Our major contribution is to contrive a caching strategy for enhanced performance in massive connection communications without knowing the content popularity. Numerical studies are performed to show that the proposed method acquires apparent performance gain over random caching in terms of average spectral efficiency and bit-error rate of the network.