Abstract
State-of-the-art light field (LF) image coding solutions, usually, rely in one of two LF data representation formats: Lenslet or 4D LF. While the Lenslet data representation is a more compact version of the LF, it requires additional camera metadata and processing steps prior to image rendering. On the contrary, 4D LF data, consisting of a stack of sub-aperture images, provides a more redundant representation requiring, however, minimal side information, thus facilitating image rendering. Recently, JPEG Pleno guidelines on objective evaluation of LF image coding defined a processing chain that allows to compare different 4D LF data codecs, aiming to facilitate codec assessment and benchmark. Thus, any codec that does not rely on the 4D LF representation needs to undergo additional processing steps to generate an output comparable to a reference 4D LF image. These additional processing steps may have impact on the quality of the reconstructed LF image, especially if color subsampling format and bit depth conversions have been performed. Consequently, the influence of these conversions needs to be carefully assessed as it may have a significant impact on a comparison between different LF codecs. Very few in-depth comparisons on the effects of using existing LF representation have been reported. Therefore, using the guidelines from JPEG Pleno, this paper presents an exhaustive comparative analysis of these two LF data representation formats in terms of LF image coding efficiency, considering different color subsampling formats and bit depths. These comparisons are performed by testing different processing chains to encode and decode the LF images. Experimental results have shown that, in terms of coding efficiency for different color subsampling formats, the Lenslet LF data representation is more efficient when using YUV 4:4:4 with 10 bit/sample, while the 4D LF data representation is more efficient when using YUV 4:2:0 with 8 bit/sample. The “best” LF data representation, in terms of coding efficiency, depends on several factors which are extensively analyzed in this paper, such as the objective metric that is used for comparison (e.g., average PSNR-Y or average PNSR-YUV), the type of LF content, as well as the color format. The maximum objective quality is also determined, by evaluating the influence of each block from each processing chain in the objective quality of the reconstructed LF image. Experimental results show that, when the 4D LF data representation is not used the maximum achieved objective quality is lower than 50 dB, in terms of average PSNR-YUV.
Highlights
Light field (LF) imaging technology allows the acquisition of radiance data from the light rays hitting the camera’s sensor, and their angular information
For each LF data representation two color formats, YUV 4:4:4 10-bit and YUV 4:2:0 8-bit, are used in the codec block of the processing chain. With these combinations of LF data representation and color formats, the effects of the suggested processing chain are evaluated in terms of objective results
The codec used in these experimental tests is the HM-16.9 implementation of HEVC-RExt [25], that may use different coding profiles, depending on the combination of the LF data representation and color format
Summary
Light field (LF) imaging technology allows the acquisition of radiance data from the light rays hitting the camera’s sensor, and their angular information. The Lenslet data representation requires minimum pre-processing before the encoding step, in order to properly render the images from the decoded LF image, camera metadata is required [4], [5]. To compare different coding techniques when encoding LF images that rely on different data representations and color formats, JPEG Pleno [20] provided a processing chain for objective comparison [21], [22]. Using the JPEG Pleno guidelines, an exhaustive comparative study is presented for both LF data representations, in terms of LF image coding efficiency and use of different color formats. Each combination of LF data representation and color format needs a specific processing chain in order to be objectively compared with a reference LF image. The remainder of this paper is organized as follows: in Section 2 both LF data representations are described, Section 3 describes the processing chains required for each combination of LF data representations and color formats, Section 4 presents the experimental results and, Section 5 concludes the paper
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
