Viewport-Aware Deep Reinforcement Learning Approach for 360$^\circ$ Video Caching

Abstract

360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> video is an essential component of VR/AR/MR systems that provides immersive experience to the users. However, 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> video is associated with high bandwidth requirements. The required bandwidth can be reduced by exploiting the fact that users are interested in viewing only a part of the video scene and that users request viewports that overlap with each other. Motivated by the findings of our recent works where the benefits of caching video tiles at edge servers instead of caching entire 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> videos were shown, in this paper, we introduce the concept of virtual viewports that have the same number of tiles with the original viewports. The tiles forming these viewports are the most popular ones for each video and are determined by the users&#x2019; requests. Then, we propose a reactive caching scheme that assumes unknown videos&#x2019; and viewports&#x2019; popularity. Our scheme determines which videos to cache as well as which is the optimal virtual viewport per video. Virtual viewports permit to lower the dimensionality of the cache optimization problem. To solve the problem, we first formulate the content placement of 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> videos in edge cache networks as a Markov Decision Process (MDP), and then we determine the optimal caching placement using the Deep Q-Network (DQN) algorithm. The proposed solution aims at maximizing the overall quality of the 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> videos delivered to the end-users by caching the most popular 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> videos at base quality along with a virtual viewport in high quality. We extensively evaluate the performance of the proposed system and compare it with that of known systems such as Least Frequently Used (LFU), Least Recently Used (LRU), First In First Out (FIFO), over both synthetic and real 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> video traces. The results reveal the large benefits coming from reactive caching of virtual viewports instead of the original ones in terms of the overall quality of the rendered viewports, the cache hit ratio, and the servicing cost.