Asymmetric and selective object rendering for optimized Cloud Mobile 3D Display Gaming user experience

Abstract

Cloud Mobile 3D Display Gaming has been recently proposed where 3D video rendering and encoding are performed on cloud servers, with the resulting 3D video streamed wirelessly to mobile devices with 3D displays. This approach has the advantage of relieving high computation, power and storage requirements of 3D display gaming from mobile devices, while enabling game developers to focus on a single rich version of the game which can be experienced from any mobile device and platform. However, it is challenging to stream 3D video over dynamically fluctuating and often constrained mobile networks. In this paper, we propose a novel technique called Asymmetric and Selective Object Rendering (ASOR) which proves to be more powerful than previous solutions for Cloud based Mobile 3D display gaming. Specifically, this technique will enable rendering engine to intelligently decide whether or not to render an individual object and how good the corresponding texture detail will be if rendered, and the settings can be asymmetric for two views. Thus, unimportant objects can trade quality for reduced bitrate while important objects can remain high quality so that the overall user experience is optimized given certain bandwidth constraints. To quantitatively measure the user experience and bitrate by applying different rendering settings, we develop a user experience model and a bitrate model. We further propose an optimization algorithm which uses the above two models to automatically decide the optimal rendering settings for left view and right view to ensure the best user experience given the network conditions. Experiments conducted using real 4G-LTE network profiles on commercial cloud service with different genres of games demonstrate significant improvement in user experience when the proposed optimization algorithm is applied.