Abstract
Light field imaging based on microlens arrays-a.k.a. holoscopic, plenoptic, and integral imaging-has currently risen up as a feasible and prospective technology for future image and video applications. However, deploying actual light field applications will require identifying more powerful representations and coding solutions that support arising new manipulation and interaction functionalities. In this context, this paper proposes a novel scalable coding solution that supports a new type of scalability, referred to as field-of-view scalability. The proposed scalable coding solution comprises a base layer compliant with the High Efficiency Video Coding (HEVC) standard, complemented by one or more enhancement layers that progressively allow richer versions of the same light field content in terms of content manipulation and interaction possibilities. In addition, to achieve high-compression performance in the enhancement layers, novel exemplar-based interlayer coding tools are also proposed, namely: 1) a direct prediction based on exemplar texture samples from lower layers and 2) an interlayer compensated prediction using a reference picture that is built relying on an exemplar-based algorithm for texture synthesis. Experimental results demonstrate the advantages of the proposed scalable coding solution to cater to users with different preferences/requirements in terms of interaction functionalities, while providing better rate-distortion performance (independently of the optical setup used for acquisition) compared to HEVC and other scalable light field coding solutions in the literature.
Highlights
THE recent advances in optical and sensor manufacturing allow having richer forms of visual data, where the spatial information about the three-dimensional (3D) scene is represented and angular viewing direction – the socalled four-dimensional (4D) light field/radiance sampling [1].In the context of Light Field (LF) imaging technologies, the approach based on a single-tier camera equipped with a Microlens Array (MLA) [2] has become a promising approach, being applicable in many different areas of research, such as 3D television [3], richer photography capturing [4], [5], biometric recognition [6], and medical imaging [7].Manuscript received March 24, 2018
Experimental results demonstrate the advantages of the proposed scalable coding solution to cater for users with different preferences/requirements in terms of interaction functionalities, while providing better rate-distortion performance compared to High Efficiency Video Coding (HEVC) and other scalable light field coding solutions in the literature
The advantages of enabling interactive media applications has been previously studied in the literature for a large range of media modalities, such as: i) interactive streaming of high resolution images [10]; ii) interactive multiview video streaming [11], [12]; and iii) interactive streaming of light field images captured by high density camera-arrays [13]
Summary
THE recent advances in optical and sensor manufacturing allow having richer forms of visual data, where the spatial information about the three-dimensional (3D) scene is represented and angular viewing direction – the socalled four-dimensional (4D) light field/radiance sampling [1]. The challenge to provide a LF representation with convenient spatial resolution and viewing angles requires handling a huge amount of data and, efficient coding becomes of utmost importance Another key requirement when designing an efficient LF representation and coding solution is to facilitate future interactive LF media applications with the new manipulation functionalities supported by the LF content. The advantages of enabling interactive media applications has been previously studied in the literature for a large range of media modalities, such as: i) interactive streaming of high resolution images [10]; ii) interactive multiview video streaming [11], [12]; and iii) interactive streaming of light field images captured by high density camera-arrays [13] In this context, standardized LF representation and coding solutions are still in an early stage of development, various LF coding solutions have been already proposed in the literature
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
