MADRL-Based Joint Edge Caching and Bitrate Selection for Multicategory 360° Video Streaming

Abstract

360-degree video streaming has gained increasing attraction in the current popular VR, AR, and MR applications, which can provide users with an immersive experience. In tile-based 360-degree video streaming, edge caching and bitrate selection strategies are jointly designed to improve users’ Quality of Experience (QoE), which incorporates video quality and rebuffer. However, the existing QoE-driven approaches use a unified QoE function to guide the decisions of edge caching and bitrate selection, which neglect the impact of video quality and rebuffer on different categories of 360-degree videos, thus failing to provide high average QoE for users. In this paper, we propose a MADRL-based joint edge caching and bitrate selection strategy for multicategory 360-degree video streaming to improve users’ average QoE. The key idea is to employ different edge caching and bitrate selection strategies for different video categories to enable fine-grained performance optimization. Based on multicategory 360-degree video streaming, we first model a joint edge caching and bitrate selection problem as a multi-agent cooperative Markov decision process with the goal of maximizing users’ average QoE. Next, an FoV-aware multi-agent soft actor-critic (FA-MASAC) algorithm is designed to help agents collaboratively learn optimal edge caching and bitrate selection decisions in a distributed way, in which each video category is treated as an agent. Finally, experimental results on real-world datasets show that our proposed strategy can greatly benefit users’ average QoE compared to existing strategies.