NA-Caching: An Adaptive Content Management Approach Based on Deep Reinforcement Learning

Abstract

Video streaming is a dominant application over today's Internet. The current mainstream video streaming solution is to utilize the services of a Content Delivery Network (CDN) provider. By replicating video content closer to the network edge, caching provides an effective mechanism for alleviating the demand for massive bandwidth for the Internet backbone. It reduces the network traffic and capital expense for streaming the video content, and in the meantime, enhance Internet's Quality of Service (QoS). In this paper, we propose a neural adaptive caching approach, named NA-Caching, for helping cache learn to make caching decisions from its own experiences rather than a specific mathematical model, in a way similar to how a human being learns a new skill (e.g. cycling, swimming). NA-Caching leverages the benefits of the Recurrent Neural Network (RNN) as well as the Deep Reinforcement Learning (DRL) to maximize the cache efficiency by jointly learning request features, caching space dynamics and making decisions. Specifically, we utilize Gated Recurrent Unit (GRU) to characterize the evolving features of the dynamic requests and caching space. Moreover, the above GRU-based representation network is integrated into a Deep Q-Network (DQN) framework for making adaptive caching decisions online. To evaluate the performance of the proposed approach, we conduct extensive experiments on anonymized real-world traces from a video provider. The results demonstrate that our algorithm significantly outperform several candidate methods.