Guest Editorial Immersive Video Coding and Transmission

Abstract

Immersive media are gaining in popularity today, and significant efforts are being undertaken in academia and industry to explore its immanent new scientific and technological challenges. There are significant activities in industry and standardization to provide enablers for production, coding, transmission, and consumption of this type of media and the new user experiences it enables. In terms of standardization, the topic has triggered multiple activities in the areas of systems, 3D graphics, audio, image, and video. The technological roadmap foresees an evolution from consumption of the visual media with three degrees of freedom (3DoF), i.e. the ability to look around at a fixed viewing position in an observed scene) to 3DoF+, enabling further limited modifications of the viewing position, and to different variants of six degrees of freedom (6DoF), allowing the user not only to look around but also to move around in the observed scene. Different terminology is used in various communities, referring to immersive or omnidirectional media, virtual reality (VR), or 360° video. While the coded representation of audio-visual media for 6DoF remains a field of active research, 3DoF technologies have sufficient maturity to progress towards specification in near-term standards and recommendations. At the video codec level, this includes coding of 2D and 3D virtual reality (VR) / 360° content using the HEVC standard (Rec. ITU-T H.265 | ISO/IEC 23008-2) as the initial step, with new Supplemental Enhancement Information (SEI) messages for omnidirectional video. For storage and delivery, the Omnidirectional Media Format (OMAF) (ISO/IEC 23090-2) provides a set of consistent enablers for download and over-the-top streaming of 3DoF content. At the same time, immersive still image coding formats are currently developed in JPEG Pleno.