Predicting Long-Term Field of View in 360-Degree Video Streaming

Abstract

Many virtual reality (VR) and augmented reality (AR) applications are closely associated with the streaming of 360-degree videos. The user only views a portion of the 360- degree video within his field of view (FoV). Thus, long-term prediction of the user's future FoV can save bandwidth resources in on-demand video streaming, minimize video rebuffering time caused by significant bandwidth fluctuations in the network and reduce unnecessary delays between the server and the client. In this article, we predict the target user's FoV based on the user's past FoV trajectory as well as the other users' future FoV trajectory. Firstly, we propose an eye-gaze-based heatmap to describe the historical movement trajectory of the user's viewport more accurately. Then, we use a method based on other user's eye gaze positions to select similar users. Finally, we introduce the squeeze-and-excitation (SE) network and U-Net (SE-Unet) to combine the historical trajectory of the target user with the future trajectory of other users to predict long-term FoV. Experimental results show that the prediction model proposed in our research is more accurate and durable.