Collaborative Edge Caching and Transcoding for 360° Video Streaming Based on Deep Reinforcement Learning

Abstract

360° video streaming provides an immersive viewing experience, but demands high bitrate transmission with low latency. To accommodate the fluctuating network condition and different user requirements, 360° videos are streamed with different quality versions, which makes the bandwidth more demanding. Edge servers with limited resources can be utilized to cache and transcode video contents near users to support 360° video streaming. The edge caching strategy and the transcoding strategy are correlated, as the cached contents limit the transcoding operations, while transcoding extends the use of cached contents at the cost of delay and provides new contents for possible caching. In this article, we consider the problem of collaborative edge transcoding and caching in an edge cluster for tile-based 360° video streaming, which aims to jointly allocate storage and computing resources to reduce quality mismatch level, delay, and transmission cost. The formulated problem is complicated considering dynamic viewers’ Field of View (FoV), video popularity, different quality versions, future user requests, and possible FoV prediction errors. To solve this problem, the process of collaborative transcoding and caching is modeled as a Markov decision process (MDP). Then, a model-free deep reinforcement learning approach, the deep deterministic policy gradient (DDPG), is used to obtain the caching replacement and computing power allocation strategy. Since the action space for caching replacement and power allocation is large, an FoV-guided scheme has been designed to speed the training of the DDPG agent. The simulation results with real viewing traces show that the proposed scheme can increase the cache hit ratio, reduce transmission cost, and improve users’ viewing experience effectively.