VMP360

Abstract

Containing full panoramic content in a single frame and providing immersive experience for users, 360° video has attracted great attention in industry and academia. Viewport-driven tiling schemes have been introduced in 360° video processing to provide high-quality video streaming. However, treating viewport as traditional streaming screen results in frequently rebuffer or quality distortion, leading to poor Quality of Experience (QoE) of schemes. In this paper, we propose Viewpoint Movement Perception 360° Video Streaming (VMP360), an adaptive 360° video streaming system that utilizes unique factors of 360° video perception quality of users to improve the overall QoE. By studying the relative moving speed and depth difference between the viewpoint and other content, the system evaluates the perceived quality distortion based on optical flow estimation. Taking QoE into account, a novel 360° video quality evaluation metric is defined as Optical-flow-based Peak Signal-to-Noise Ratio (OPSNR). Appling OPSNR to tiling process, VMP360 proposes a versatile-size tiling scheme, and further Reinforcement Learning (RL) is used to realize the Adaptive Bit Rate (ABR) selection of tiles. VMP360 is evaluated through the client-server streaming system with two prior schemes Pano and Plato. Statistics show that the proposed scheme can improve the quality of 360° video by 10.1% while maintaining same rebuffer ratio compared with the Pano and Plato, which confirms that VMP360 can provide a promising high QoE for 360° video streaming. The code of a prototype can be found in https://github.com/buptexplorers/OFB-VR.