Saliency-driven rate-distortion optimization for 360-degree image coding

Abstract

360-degree images allow an immersive experience. They offer multiple views of a scene and the viewpoint can be selected by the user. However, the huge amount of data that is necessary for real-time transmission of 360-degree image and video requires efficient coding techniques, particularly for virtual reality (VR) and augmented reality (AR) applications. The viewer is only interested in a part of the scene so compressing the entire scene with equal quality is inefficient. This study initially constructs a saliency model of the 360-degree image and then a visual attention guided coding scheme is developed using a predicted saliency map. For saliency prediction, two methods of saliency prediction are used and the results are fused, to address the problem of geometry distortion in the ERP (Equirectangular Projection) format. A smoothing-based optimization is then realized in the spherical domain to improve the saliency map. Using the saliency map of the 360-degree image, the distortion of the rate-distortion optimization is modified to ensure a better visual experience. The experimental results show that the viewports of greatest interest are rendered with the highest quality and there is a maximum of 14.33% reduction in the bitrate when the quality measurement is performed in these regions.