Video processing for panoramic streaming using HEVC and its scalable extensions

Y Sánchez De La Fuente,T Schierl,R Skupin

doi:10.1007/s11042-016-4097-4

Y Sánchez De La Fuente, T Schierl + Show 1 more

Open Access

https://doi.org/10.1007/s11042-016-4097-4

Copy DOI

Abstract

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) is transmitted from a high-spatial resolution video, i.e. a video covering a very “wide-angle” (much larger than the human field-of-view – e.g. 360°). Some transport schemes for panoramic video delivery have been proposed and demonstrated within the past decade, which allow users to navigate interactively within the high-resolution videos. With the recent advances of head mounted displays, consumers may soon have immersive and sufficiently convenient end devices at reach, which could lead to an increasing demand for panoramic video experiences. The solution proposed within this paper is built upon tile-based panoramic streaming, where users receive a set of tiles that match their RoI, and consists in a low-complexity compressed domain video processing technique for using H.265/HEVC and its scalable extensions (H.265/SHVC and H.265/MV-HEVC). The proposed technique generates a single video bitstream out of the selected tiles so that a single hardware decoder can be used. It overcomes the scalability issue of previous solutions not using tiles and the battery consumption issue inherent of tile-based panorama streaming, where multiple parallel software decoders are used. In addition, the described technique is capable of reducing peak streaming bitrate during changes of the RoI, which is crucial for allowing a truly immersive and low latency video experience. Besides, it makes it possible to use Open GOP structures without incurring any playback interruption at switching events, which provides a better compression efficiency compared to closed GOP structures.

Highlights

Panoramic streaming is a specific case of video streaming, in which an arbitrary Region-ofInterest (RoI) of a high-spatial resolution video is transmitted
It can be seen that in particular the critical peak bitrates within the RoI change interval are considerably reduced using the proposed Multiview Generated Reference Pictures (MGRP) technique, which is sensible keeping in mind that much of the unnecessary intra-coded data is omitted from the bitstream
The quality of the Random Access Skip Leading (RASL) pictures when using the ML-Generated Reference Pictures (GRP) has not been subjectively measured, our assessment by looking at the resulting videos is that the technique performs very good

Summary

Introduction

Panoramic streaming is a specific case of video streaming, in which an arbitrary Region-ofInterest (RoI) of a high-spatial resolution video is transmitted. Taking into account that there exist techniques and commercial products that allow capturing 360° video in real time [28], by stitching multiple HD views from multiple cameras, and the recent market availability of plenty of head mounted consumer displays such as the Oculus Rift [6], Samsung Gear VR [7] or Google Cardboard [8], we envision that interactive panoramic streaming will be a popular application in a few years . Consumers may soon have immersive and sufficiently convenient end devices and content at reach, which could lead to an increasing demand of panoramic video experiences

Objectives

Methods

Results

Conclusion