Computational Integral Imaging System Research Articles

Computational Large Field-of-View RGB-D Integral Imaging System.

The integral imaging system has received considerable research attention because it can be applied to real-time three-dimensional image displays with a continuous view angle without supplementary devices. Most previous approaches place a physical micro-lens array in front of the image, where each lens looks different depending on the viewing angle. A computational integral imaging system with a virtual micro-lens arrays has been proposed in order to provide flexibility for users to change micro-lens arrays and focal length while reducing distortions due to physical mismatches with the lens arrays. However, computational integral imaging methods only represent part of the whole image because the size of virtual lens arrays is much smaller than the given large-scale images when dealing with large-scale images. As a result, the previous approaches produce sub-aperture images with a small field of view and need additional devices for depth information to apply to integral imaging pickup systems. In this paper, we present a single image-based computational RGB-D integral imaging pickup system for a large field of view in real time. The proposed system comprises three steps: deep learning-based automatic depth map estimation from an RGB input image without the help of an additional device, a hierarchical integral imaging system for a large field of view in real time, and post-processing for optimized visualization of the failed pickup area using an inpainting method. Quantitative and qualitative experimental results verify the proposed approach’s robustness.

Improved integral imaging based image copyright protection algorithm using 3-D computational integral imaging pickup and super-resolution reconstruction technique

Cellular automata transform (CAT) based watermarking method can provide multi-transform planes for embedding watermark. In this paper, to overcome conventional transform-based watermarking methods have only one transform plane for watermark embedding, we propose a CAT-based watermarking system, in which the three-dimensional (3-D) watermark data is decomposed by the computational integral imaging (CII) technique and be embedded into the CAT domain. However, in the 3-D watermark reconstruction process, the computational integral imaging reconstruction (CIIR) technique in CII system that can degrade the quality of the extracted watermark due to the pixel overlapping. We introduce the super-resolution reconstruction technique that can effectively improve this problem. Some experiments are carried out to verify the performance of the CAT-based watermarking scheme. From the experimental results, the proposed watermarking system can provide good imperceptibility and high robustness.

Color 3D Image Encryption Technique by Combined Use of Computational Integral Imaging Joint Image Scrambling Technique

In this paper, we propose a novel three-dimensional (3D) color image encryption scheme based on computational integral imaging (CII) and Arnold transform technique. An elemental image array (EIA) is firstly captured by recording the light rays emanating from 3D image through a lenslet array. Then, the captured color EIA is decomposed into three independent channels: red, green and blue, and each channel is encrypted by Arnold transform. In the reconstruction process, the pseudo-inverse filter is used to improve quality of the reconstructed 3D color image in CII system. Numerical simulations have been done to prove the validity and the security of the proposed 3D color image encryption method.

3D Integral Imaging Encryption Using a Depth-Converted Elemental Image Array

We propose a three-dimensional (3D) image encryption method based on the modified computational integral imaging (CII) technique with the smart pixel mapping (SPM) algorithm, which is introduced for reconstructing orthoscopic 3D images with improved image quality. The depth-converted two-dimensional (2D) elemental image array (EIA) is firstly obtained by SPM-based CII system, and then the 2D EIA is encrypted by Fibonacci transform for 3D image encryption. Compared with conventional encryption methods based on integral imaging (II), the proposed method enables us to reconstruct orthoscopic 3D images at long distance. In addition, the qualities of the reconstructed plane images are enhanced by applying the SPM and pixel average algorithm (PAA) in CII. To show the usefulness of the proposed method, we carry out the preliminary experiments and present the experimental results.

Accelerated Generation Algorithm for an Elemental Image Array Using Depth Information in Computational Integral Imaging

In this paper, an accelerated generation algorithm to effectively generate an elemental image array in computational integral imaging system is proposed. In the proposed method, the depth information of 3D object is extracted from the images picked up by a stereo camera or depth camera. Then, the elemental image array can be generated by using the proposed accelerated generation algorithm with the depth information of 3D object. The resultant 3D image generated by the proposed accelerated generation algorithm was compared with that the conventional direct algorithm for verifying the efficiency of the proposed method. From the experimental results, the accuracy of elemental image generated by the proposed method could be confirmed.

Axially moving a lenslet array for high-resolution 3D images in computational integral imaging

This paper presents a new high-resolution computational integral imaging system employing a pickup with the axial movement of a lenslet array and a computation reconstruction algorithm with pixel-to-pixel mapping. In the proposed method, a lenslet array and its image sensor are moved together along the z-axis direction (or axial direction) and a series of elemental image arrays are obtained while moving. The elemental image arrays are then applied to pixel-to-pixel mapping without interpolation for the reconstruction of 3D slice images. Also, an analysis of the proposed reconstruction method is provided. To show the usefulness of the proposed method, experiments are conducted. The results indicate that the proposed method is superior to the existing method such as MALT in terms of image quality.

Fast 3D Computational Integral Imaging Using Graphics Processing Unit

In computational integral imaging (II), the elemental images are processed on serial processors to reconstruct one plane (slice) of the 3D scene. In this paper, we present a fast three-dimensional (3D) integral imaging system via a graphics processing unit (GPU) which allows parallel processing with multiple processors. We show that it can significantly accelerate 3D scene reconstruction in II using the GPU based stream-processing model. The streaming version of the ray back propagation algorithm with lookup table is presented. It is demonstrated that the ray back propagation algorithm with a lookup table for the 3D scene reconstruction in II to be processed on parallel processors may greatly improve computational speed while requiring minimally larger memory space as compared with CPU sequential computing. Experimental results verify the feasibility for parallel implementation of 3D integral imaging. To the best of our knowledge, this is the first study on achieving a 3D computational integral imaging system using GPU computing with high parallelism.

Computational integral imaging-based 3D digital watermarking scheme using cellular automata transform and maximum length cellular automata

A novel 3D watermarking algorithm by combining use of computational integral imaging (CII) and cellular automata transform (CAT) is proposed in this paper. In this proposed scheme, first, the original image signal is decomposed into three resolution levels by using the level-3 2D CAT, and meanwhile, the middle-frequency domains are obtained. Then, an elemental images (EIs) array is generated by recording the information of rays of light coming from an object through a pinhole array in the CII system. The EIs array is encrypted by linear maximum-length cellular automata as the encrypted watermark embedded into the CAT middle-frequency domains. Finally, the watermarked image is obtained by using the level-3 2D inverse CAT. To verify the usefulness of the proposed algorithm, we carry out the computational experiments and present the experimental results for various attacks. Experimental results show that this proposed watermarking system provides excellent results in unobtrusiveness and robustness.

High performance computational integral imaging system using multi-view video plus depth representation

Integral imaging is an attractive auto-stereoscopic three-dimensional (3D) technology for next-generation 3DTV. But its application is obstructed by poor image quality, huge data volume and high processing complexity. In this paper, a new computational integral imaging (CII) system using multi-view video plus depth (MVD) representation is proposed to solve these problems. The originality of this system lies in three aspects. Firstly, a particular depth-image-based rendering (DIBR) technique is used in encoding process to exploit the inter-view correlation between different sub-images (SIs). Thereafter, the same DIBR method is applied in the display side to interpolate virtual SIs and improve the reconstructed 3D image quality. Finally, a novel parallel group projection (PGP) technique is proposed to simplify the reconstruction process. According to experimental results, the proposed CII system improves compression efficiency and displayed image quality, while reducing calculation complexity.

Effective reconstruction of a partially occluded 3-D target by using a pixel restoration scheme in computational integral-imaging

In this paper, we propose an effective approach for visibility-enhanced reconstruction of a partially occluded 3-D target by using the pixel-restoration method in the computational integral-imaging system. In the proposed method, elemental images (EIs) of the 3-D target that are partially occluded by a foreground object are picked up and transformed into sub-images (SIs). By using the block-matching algorithm then, the occluded target regions of each sub-image are estimated and removed. After that, the missing pixels in each sub-image are re-established by using the proposed pixel-restoration method. Finally, these restored SIs are converted back into the newly modified EIs, from which the resolution-enhanced target can be reconstructed with the computational integral-imaging reconstruction (CIIR) method. Successful experiments with the test objects confirm the feasibility of the proposed method.

Three-dimensional polarimetric computational integral imaging

In this paper, we propose a novel 3D polarimetric computational integral imaging system by using polarization diversity of objects under natural illumination conditions. In the system, the measured Stokes polarization parameters are utilized to generate degree of polarization images of a 3D scene. Based on degree of polarization images and original 2D images, we utilize a modified computational reconstruction method to perform 3D polarimetric image reconstruction. The system may be used to detect or classify objects with distinct polarization signatures in 3D space. Experimental results also show the proposed system may mitigate the effect of occlusion in 3D reconstruction.

Enhanced computational integral imaging system for partially occluded 3D objects using occlusion removal technique and recursive PCA reconstruction

In this paper, we propose an enhanced computational integral imaging system by both eliminating the occlusion in the elemental images recorded from the partially occluded 3D object and recovering the entire elemental images of the 3D object. In the proposed system, we first obtain the elemental images for partially occluded object using computational integral imaging system and it is transformed to sub-images. Then we eliminate the occlusion within the sub-images by use of an occlusion removal technique. To compensate the removed part from occlusion-removed sub-images, we use a recursive application of PCA reconstruction and error compensation. Finally, we generate the entire elemental images without a loss from the newly reconstructed sub-images and perform the process of object recognition. To show the usefulness of the proposed system, we carry out the computational experiments for face recognition and its results are presented. Our experimental results show that the proposed system might improve the recognition performance dramatically.

Improved analysis on the signal property of computational integral imaging system

In this paper, we introduce an improved signal analysis of the computational integral imaging (CII) system having a pickup process of three-dimensional object and a volumetric computational reconstruction (VCR) process. We propose a signal model for the CII system. From the signal model and its analysis, we can define a granular noise caused by the non-uniform overlapping. We also analyze the characteristics of the granular noise. According to our model and analysis, there is a condition that the granular noise cancels out. To show the feasibility of our model, the preliminary experiments are carried out and the result is presented. This is the first time, to our knowledge, that a signal model for the analysis of CII systems is provided.