Raw Light Field Research Articles

Quantitative real-time phase microscopy for extended depth-of-field imaging based on the 3D single-shot differential phase contrast (ssDPC) imaging method.

Optical diffraction tomography (ODT) is a promising label-free imaging method capable of quantitatively measuring the three-dimensional (3D) refractive index distribution of transparent samples. In recent years, partially coherent ODT (PC-ODT) has attracted increasing attention due to its system simplicity and absence of laser speckle noise. Quantitative phase imaging (QPI) technologies represented by Fourier ptychographic microscopy (FPM), differential phase contrast (DPC) imaging and intensity diffraction tomography (IDT) need to collect several or hundreds of intensity images, which usually introduce motion artifacts when shooting fast-moving targets, leading to a decrease in image quality. Hence, a quantitative real-time phase microscopy (qRPM) for extended depth of field (DOF) imaging based on 3D single-shot differential phase contrast (ssDPC) imaging method is proposed in this research study. qRPM incorporates a microlens array (MLA) to simultaneously collect spatial information and angular information. In subsequent optical information processing, a deconvolution method is used to obtain intensity stacks under different illumination angles in a raw light field image. Importing the obtained intensity stack into the 3D DPC imaging model is able to finally obtain the 3D refractive index distribution. The captured four-dimensional light field information enables the reconstruction of 3D information in a single snapshot and extending the DOF of qRPM. The imaging capability of the proposed qRPM system is experimental verified on different samples, achieve single-exposure 3D label-free imaging with an extended DOF for 160 µm which is nearly 30 times higher than the traditional microscope system.

Snapshot depth–spectral imaging based on image mapping and light field

Depth–spectral imaging (DSI) is an emerging technology which can obtain and reconstruct the spatial, spectral and depth information of a scene simultaneously. Conventionally, DSI system usually relies on scanning process, multi-sensors or compressed sensing framework to modulate and acquire the entire information. This paper proposes a novel snapshot DSI architecture based on image mapping and light field framework by using a single format detector. Specifically, we acquire the depth – spectral information in two steps. Firstly, an image mapper is utilized to slice and reflect the first image to different directions which is a spatial modulation processing. The modulated light wave is then dispersed by a direct vision prism. After re-collection, the sliced dispersed light wave is recorded by a light field sensor. Complimentary, we also propose a reconstruction strategy to recover the spatial depth – spectral hypercube effectively. We establish a mathematical model to describe the light wave distribution on every optical facet. Through simulations, we generate the aliasing raw spectral light field data. Under the reconstruction strategy, we design an algorithm to recover the hypercube accurately. Also, we make an analysis about the spatial and spectral resolution of the reconstructed data, the evaluation results conform the expectation.

End-to-End Residual Network for Light Field Reconstruction on Raw Images and View Image Stacks

Light field (LF) technology has become a focus of great interest (due to its use in many applications), especially since the introduction of the consumer LF camera, which facilitated the acquisition of dense LF images. Obtaining densely sampled LF images is costly due to the trade-off between spatial and angular resolutions. Accordingly, in this research, we suggest a learning-based solution to this challenging problem, reconstructing dense, high-quality LF images. Instead of training our model with several images of the same scene, we used raw LF images (lenslet images). The raw LF format enables the encoding of several images of the same scene into one image. Consequently, it helps the network to understand and simulate the relationship between different images, resulting in higher quality images. We divided our model into two successive modules: LFR and LF augmentation (LFA). Each module is represented using a convolutional neural network-based residual network (CNN). We trained our network to lessen the absolute error between the novel and reference views. Experimental findings on real-world datasets show that our suggested method has excellent performance and superiority over state-of-the-art approaches.

All-In-Focus Polarimetric Imaging Based on an Integrated Plenoptic Camera with a Key Electrically Tunable LC Device.

In this paper, a prototyped plenoptic camera based on a key electrically tunable liquid-crystal (LC) device for all-in-focus polarimetric imaging is proposed. By using computer numerical control machining and 3D printing, the proposed imaging architecture can be integrated into a hand-held prototyped plenoptic camera so as to greatly improve the applicability for outdoor imaging measurements. Compared with previous square-period liquid-crystal microlens arrays (LCMLA), the utilized hexagonal-period LCMLA has remarkably increased the light utilization rate by ~15%. Experiments demonstrate that the proposed imaging approach can simultaneously realize both the plenoptic and polarimetric imaging without any macroscopic moving parts. With the depth-based rendering method, both the all-in-focus images and the all-in-focus degree of linear polarization (DoLP) images can be obtained efficiently. Due to the large depth-of-field advantage of plenoptic cameras, the proposed camera enables polarimetric imaging in a larger depth range than conventional 2D polarimetric cameras. Currently, the raw light field images with three polarization states including I0 and I60 and I120 can be captured by the proposed imaging architecture, with a switching time of several tens of milliseconds. Some local patterns which are selected as interested target features can be effectively suppressed or obviously enhanced by switching the polarization state mentioned. According to experiments, the visibility in scattering medium can also be apparently improved. It can be expected that the proposed polarimetric imaging approach will exhibit an excellent development potential.

Metric calibration of unfocused plenoptic cameras for three-dimensional shape measurement

A metric calibration method for light-field three-dimensional (3-D) reconstruction based on plenoptic disk features is presented. Each of these features that relates to a single 3-D point in free physical space could be directly extracted from the raw light field. By establishing the point-to-feature correspondence, the optical parameters of unfocused plenoptic camera would be estimated through nonlinear optimization, which could then be exploited to metrically generate the 3-D point cloud of an object. For verification, all detected plenoptic disk features are reconstructed to 3-D space and then light-field reprojection is performed. In addition, to demonstrate the performance of the proposed method, comparative experiments are carried out on a commercial unfocused plenoptic camera system. Then, the acquired datasets are processed using the published method based on plenoptic disk features and our proposed calibration method, respectively. Experimental results show that the proposed calibration algorithm is more reliable and faster than the existing published one, especially for small field-of-view applications.

Research on high-resolution imaging technology based on light field manipulation for a lenslet-based plenoptic camera.

In this paper, an aberration correction method for an extended target is proposed to solve the problem of the lenslet-based plenoptic camera not imaging clearly under the influence of aberrations. We propose a light field manipulation method to improve performance of the light field imaging system. The principle of this method is that the sub-aperture images extracted from the raw light field image are offset when the light field imaging system is affected by aberrations, and the symmetrical arrangement of the sub-aperture image array is destroyed. By repairing the symmetrical arrangement of the sub-aperture image array, the influence of phase aberrations on the imaging system can be eliminated, and the resolution of the plenoptic camera can be improved. We use an image correlation algorithm to process the sub-aperture images of the plenoptic camera, calculate and compensate each sub-aperture image's displacement caused by aberrations, and restore the symmetrical arrangement of the sub-aperture image array; then, a corrected high-resolution refocused image can be generated. In particular, this method uses only the raw light field information obtained by the plenoptic camera in a single exposure, without adding other hardware devices. Furthermore, it takes the extended target itself as the reference image, so the ideal position need not be calibrated in advance. Also, the parallax information of the sub-aperture images is retained, and the method is simple and easy to use. Numerical simulation and experimental results show that the technology proposed in this paper can work well for high-resolution imaging of a plenoptic camera with phase aberrations. This method can be potentially applied to analyze lens aberration, media-induced image distortion such as water turbulence in underwater imaging, and atmospheric turbulence in remote imaging. It may have important application prospects in the fields of astronomical object detection, remote sensing, etc.

Light field image compression based on quality aware pseudo‐temporal sequence

Currently, a raw light field image can be captured by a light field camera and decomposed into multi-views with different perceptual quality. A novel light field compression approach for a light field image is proposed based on this observation, where the raw light field image is decomposed into multi-views as pseudo-sequencing frames and reordered according to their perceptual quality. The resulting pseudo-temporal sequence is then compressed by high-efficiency video coding (HEVC) encoder. Experimental results show that the proposed method can achieve a better coding performance (up to 28% Bjontegaard delta bit rate reduction (BDBR) savings) compared with state-of-the-art reordering techniques using HEVC.

Light Field Compression on Sliced Lenslet Array

Recent advances in light field technologies is fostering the research and development of novel imaging applications. Such applications perform processing of the light field information to create new visual effects such as, for example, refocusing, perspective change, colour adjustment. Light field imaging is very data intensive compared with usual digital photographic imaging, and novel compression algorithms are needed for addressing the problem of light field storage and transmission. Raw digital light field images exhibit low spatial correlation if compared to regular images and hence the performance in terms of rate-distortion of current state-of-the-art image encoders can be superseded by devising novel image compression architectures. In this paper, an architecture for lossy compression of unfocused light field images is proposed. Raw light fields are preprocessed by demosaicing, devignetting and slicing of the raw lenslet array image. The slices are then compressed with the JPEG 2000 image coding standard. The performance of the proposed method is compared against direct application of JPEG 2000 compression on the 4D light field. The experimental analysis has been conducted under a set of different compression ratios and the obtained results show that the proposed method outperforms direct application of the reference architecture.

Light field compression on sliced lenslet array

Recent advances in light field technologies is fostering the research and development of novel imaging applications. Such applications perform processing of the light field information to create new visual effects such as, for example, refocusing, perspective change, colour adjustment. Light field imaging is very data intensive compared with usual digital photographic imaging, and novel compression algorithms are needed for addressing the problem of light field storage and transmission. Raw digital light field images exhibit low spatial correlation if compared to regular images and hence the performance in terms of rate-distortion of current state-of-the-art image encoders can be superseded by devising novel image compression architectures. In this paper, an architecture for lossy compression of unfocused light field images is proposed. Raw light fields are preprocessed by demosaicing, devignetting and slicing of the raw lenslet array image. The slices are then compressed with the JPEG 2000 image coding standard. The performance of the proposed method is compared against direct application of JPEG 2000 compression on the 4D light field. The experimental analysis has been conducted under a set of different compression ratios and the obtained results show that the proposed method outperforms direct application of the reference architecture.

Face liveness detection using a light field camera.

A light field camera is a sensor that can record the directions as well as the colors of incident rays. This camera is widely utilized from 3D reconstruction to face and iris recognition. In this paper, we suggest a novel approach for defending spoofing face attacks, like printed 2D facial photos (hereinafter 2D photos) and HD tablet images, using the light field camera. By viewing the raw light field photograph from a different standpoint, we extract two special features which cannot be obtained from the conventional camera. To verify the performance, we compose light field photograph databases and conduct experiments. Our proposed method achieves at least 94.78% accuracy or up to 99.36% accuracy under different types of spoofing attacks.