Caching Strategy Based on Deep Q-Learning in Device-to-device Scenario

Abstract

With the explosive growth of multimedia data traffic represented by video-on-demand (VOD), the spectrum resource utilization of existing wireless networks is increasingly difficult to meet the needs. To solve this problem, device-to-device (D2D) technology is proposed in the 5th generation mobile communication (5G) standard. Relevant research has realized the offloading of video data in cellular network by introducing some auxiliary caching devices (helpers) in D2D network, so as to alleviate the pressure of base station and improve the utilization of spectrum. In this paper, we propose and experiment an algorithm to find the caching strategy of helpers based on deep reinforcement learning in D2D Scenario. And a structure of D2D auxiliary caching device is designed to apply the algorithm. The caching strategy generated by this algorithm can effectively guide the caching data settings in helpers. Experiments show that the VOD system based on the proposed algorithm has a higher utilization rate of bandwidth resources and lower base station load. Compared with random algorithm, the system cache hit rate and system offload rate increased by 12% and 16%, compared with FIFO algorithm, increased by 11% and 15%, and compared with LRU algorithm, increased by 4% and 6%.