Optimizing Cloud Mobile 3D Display Gaming user experience by asymmetric object of interest rendering

Abstract

The growing popularity of auto-stereoscopic 3D displays for mobile devices, together with ubiquitous wireless networks, have fueled an increasing user expectation for rich 3D mobile multimedia experiences, including 3D display gaming. However, rendering 3D games on mobile devices requires high computational power and battery and thus may restrict users from enjoying true 3D experience for a long time. In this paper, we explore the possibility of developing Cloud Mobile 3D Display Gaming, where the 3D video rendering and encoding are performed on cloud servers, with the resulting 3D video streamed wirelessly to mobile devices with 3D displays. However, with the significantly higher bitrate requirement for 3D video, ensuring user experience may be a challenge considering the bandwidth constraints and fluctuations of mobile networks. In this paper, we propose a novel asymmetric Object of Interest (OOI) rendering approach, which adapts the rendering richness of different objects according to their importance in order to reduce the video encoding bitrate needed while maintaining a satisfactory video quality, thereby making it easier to transmit the 3D game video over wireless network. Specifically, we first develop a model to quantitatively measure the user experience by different OOI rendering settings. We also develop a model to relate the bitrate of the resulting video with the changes of different OOI Rendering settings. We further propose an optimization algorithm which uses the above two models to automatically decide the optimal OOI rendering settings for left view and right view to ensure the best user experience given the network conditions. Experiments conducted using real 4G-LTE network profiles on commercial cloud service demonstrate the improvement in user experience when the proposed optimization algorithm is applied.