An approximate dynamic programming approach for collaborative caching

Abstract

In this article, online collaborative content caching in wireless networks is studied from a network economics point of view. The cache optimization problem is first modelled as a finite horizon Markov decision process that incorporates an auto-regressive model to forecast the evolution of the content demands. The complexity of the problem grows exponentially with the system parameters, and even though a good approximation to the cost-to-go can be found, the single-stage decision problem is still NP-hard. To deal with cache optimization in industrial-size networks, a novel methodology called rolling horizon is proposed that solves the dimensionality of the problem by freezing the cache decisions for a short number of periods to construct a value function approximation. Then, to address the NP-hardness of the single-stage decision problem, two simplifications/reformulations are examined: (a) to limit the number of content replicas in the network and (b) to limit the allowed content replacements. The results show that the proposed approach can reduce the communication cost by over 84% compared to that of running least recently used updates on offline schemes in collaborative caching. The results also shed light on the trade-off between the efficiency of the caching policy and the time needed to run the online cache optimization algorithm.