Abstract
This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited, as is demonstrated. This novel hybrid data representation approach allows to simultaneously exploit four types of redundancies: i) sub-aperture image intra spatial redundancy, ii) sub-aperture image inter-view redundancy, iii) intra-micro-image redundancy, and iv) inter-micro-image redundancy between neighboring micro-images. The proposed light field coding solution allows flexibility for several types of baselines, by adaptively exploiting the most predominant type of redundancy on a coding block basis. To demonstrate the efficiency of using a hybrid representation, this paper proposes a set of efficient pixel prediction methods combined with a pseudo-video sequence coding approach, based on the HEVC standard. Experimental results show consistent average bitrate savings when the proposed codec is compared to relevant state-of-the-art benchmarks. For lenslet light field content, the proposed coding algorithm outperforms the HEVC-based pseudo-video sequence coding benchmark by an average bitrate savings of 23%. It is shown for the same light field content that the proposed solution outperforms JPEG Pleno verification models MuLE and WaSP, as these codecs are only able to achieve 11% and -14% bitrate savings over the same HEVC-based benchmark, respectively. The performance of the proposed coding approach is also validated for light fields with wider baselines, captured with high-density camera arrays, being able to outperform both the HEVC-based benchmark, as well as MuLE and WaSP.
Highlights
The light field (LF) imaging technology allows to jointly capture the scene radiance and angular information using single-tier lenslet LF cameras, i.e., with narrow baseline, or by using, for example, a high-density camera array (HDCA), i.e., with a wider baseline
The remainder of this paper is organized as follows: Section II reviews the state-of-the-art on LF coding approaches based on MIs and sub-aperture images (SAIs); Section III presents the proposed hybrid LF data representation; Section IV describes the new intra-MI prediction modes proposed in this paper; Section V presents the new inter-MI prediction modes proposed in this paper; Section VI evaluates the performance of the proposed LF coding solution against the most relevant state-of-the-art solutions; and, Section VII concludes the paper
To efficiently exploit the intra-MI redundancy within each MI, a set of pixel-based prediction methods, i.e., DC, median edge detector (MED), gradient adjusted predictor (GAP) and accurate gradient selective prediction (AGSP) were adapted to the proposed codec
Summary
The light field (LF) imaging technology allows to jointly capture the scene radiance and angular information using single-tier lenslet LF cameras, i.e., with narrow baseline, or by using, for example, a high-density camera array (HDCA), i.e., with a wider baseline. Monteiro et al.: LF Image Coding Based on Hybrid Data Representation several views from different perspectives, 2D, 3D and multiview (MV) signals can be created This imaging technology allows for interactive media applications, such as interactive MV video [6], [7], free viewpoint video streaming [8], and interactive streaming of light field images captured by HDCAs [9] or lenslet LF cameras [10]. The remainder of this paper is organized as follows: Section II reviews the state-of-the-art on LF coding approaches based on MIs and SAIs; Section III presents the proposed hybrid LF data representation; Section IV describes the new intra-MI prediction modes proposed in this paper; Section V presents the new inter-MI prediction modes proposed in this paper; Section VI evaluates the performance of the proposed LF coding solution against the most relevant state-of-the-art solutions; and, Section VII concludes the paper
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have