Energy-Efficient Joint Pushing and Caching Based on Markov Decision Process

Abstract

Listen

Translate article

Content caching provides a rare trade-off opportunity between low-cost memory and energy consumption, yet finding the optimal causal policy with low computational complexity remains a challenge. In this paper, we formulate the problem of joint pushing and caching at end users in a Markov decision process (MDP) framework with the aim of minimizing energy cost under causal knowledge of the user request delay information and strict delay constraint. We put forth a novel approach to decouple the influence of buffer occupancy and user request, allowing one to equivalently run value iteration in a much smaller state space. Furthermore, by exploiting a structural property we discovered as generalized monotonicity , we design a computationally efficient non-iterative algorithm, fast assignment of state transition, to map the optimal policy from the smaller state space back to the complete state space. This, in turn, significantly simplifies the search for optimal policy. The results attain close performance in comparison with theoretical bounds from non-causal policies, while benefiting from better time efficiency than the unadapted MDP solutions.