Microlens image sparse modelling for lossless compression of plenoptic camera sensor images

Abstract

This paper studies the lossless compressibility of raw sensor images acquired by plenoptic cameras, when optimally interpolating the microlens images in terms of already encoded microlens images. The geometrical information necessary for splitting the sensor image into projections of microlenses, together with a relatively small bitstream for encoding the raw image at the microlens centers are encoded as a first stage. The scanning order for sampling the data from the sensor follows row-by-row the approximate hexagonal lattice pattern of the microlenses, and the pixels inside each microlens are scanned in an ascending spiral order. The predictive encoding of a pixel from a microlens block uses the similarly located pixels (possibly slightly shifted) in the blocks from nine closest causal microlenses (those already encoded) and the pixels from its own microlens located in the encoded part of the spiral. A minimum description length optimal sparse predictor is designed for each microlens. The sparsity masks and prediction coefficients are encoded in a second stage and the prediction errors at every pixel are finally encoded in a third stage, in a view-by-view order (a view index being determined by the pixel's index in its block), using contexts accounting for the magnitude of errors at views already encoded. The experimental results show better performance than the JPEG 2000 image standard applied on the raw image.