Integral Imaging Reconstruction Research Articles

High-precision integral imaging 3D salient object detection and reconstruction with texture features based on E2E-TransGAN.

Under the framework of integral imaging, a high-precision 3D salient object detection and high-quality texture features reconstruction method is proposed by using the element to element-transformer and generative adversarial network (E2E-TransGAN). The proposed method fully exploits the global salient clues of the element image array to establish cross-regional element image dependency, thereby the accuracy of the salient prediction improved. Meanwhile, our proposed E2E-TransGAN algorithm stratifies the binary salient object area and the grayscale salient object edge, effectively addressing salient region prediction errors, obvious background noise, and color loss. The experimental results show that the detection accuracy of the proposed method improves to an average of 90% across various evaluation metrics. Compared with other methods, the proposed method can reconstruct 3D images with texture features, which have high values of PSNR and SSIM, at 30 and 0.95 on average, respectively. We have experimentally verified that the proposed method achieves precise 3D object salient detection and restores the detailed textures and depth information of 3D salient objects. The proposed method provides strong support for efficient and accurate medical image analysis and simplifies subsequent computer vision tasks.

Three-Dimensional Single Random Phase Encryption.

In this paper, we propose a new optical encryption technique that uses the single random phase mask. In conventional optical encryptions such as double random phase encryption (DRPE), two different random phase masks are required to encrypt the primary data. For decryption, DRPE requires taking the absolute value of the decrypted data because it is complex-valued. In addition, when key information is revealed, the primary data may be reconstructed by attackers. To reduce the number of random phase masks and enhance the security level, in this paper, we propose single random phase encryption (SRPE) with additive white Gaussian noise (AWGN) and volumetric computational reconstruction (VCR) of integral imaging. In our method, even if key information is known, the primary data may not be reconstructed. To enhance the visual quality of the decrypted data by SRPE, multiple observation is utilized. To reconstruct the primary data, we use VCR of integral imaging because it can remove AWGN by average effect. Thus, since the reconstruction depth can be another key piece of information of SRPE, the security level can be enhanced. In addition, it does not require taking the absolute value of the decrypted data for decryption. To verify the validity of our method, we implement the simulation and calculate performance metrics such as peak sidelobe ratio (PSR) and structural similarity (SSIM). In increasing the number of observations, SSIM for the decrypted data can be improved dramatically. Moreover, even if the number of observations is not enough, three-dimensional (3D) data can be decrypted by SRPE at the correct reconstruction depth.

Computational Integral Imaging Reconstruction Based on Generative Adversarial Network Super-Resolution

To improve acquisition efficiency and achieve super high-resolution reconstruction, a computational integral imaging reconstruction (CIIR) method based on the generative adversarial network (GAN) network is proposed. Firstly, a sparse camera array is used to generate an elemental image array of the 3D object. Then, the elemental image array is mapped to a low-resolution sparse view image. Finally, a lite GAN super-resolution network is presented to up-sample the low-resolution 3D images to high-resolution 3D images with realistic image quality. By removing batch normalization (BN) layers, reducing basic blocks, and adding intra-block operations, better image details and faster generation of super high-resolution images can be achieved. Experimental results demonstrate that the proposed method can effectively enhance the image quality, with the structural similarity (SSIM) reaching over 0.90, and can also reduce the training time by about 20%.

Computational Integral Imaging Reconstruction via Elemental Image Blending without Normalization

This paper presents a novel computational integral imaging reconstruction (CIIR) method using elemental image blending to eliminate the normalization process in CIIR. Normalization is commonly used in CIIR to address uneven overlapping artifacts. By incorporating elemental image blending, we remove the normalization step in CIIR, leading to decreased memory consumption and computational time compared to those of existing techniques. We conducted a theoretical analysis of the impact of elemental image blending on a CIIR method using windowing techniques, and the results showed that the proposed method is superior to the standard CIIR method in terms of image quality. We also performed computer simulations and optical experiments to evaluate the proposed method. The experimental results showed that the proposed method enhances the image quality over that of the standard CIIR method, while also reducing memory usage and processing time.

Fast Numerical Reconstruction of Integral Imaging Based on a Determined Interval Mapping

In this paper, a fast numerical reconstruction of the integral imaging based on a determined interval mapping is proposed. To reduce the computation time, the proposed method employs the determined interval mapping instead of the use of magnification. In the numerical reconstruction procedure, the acquired elemental image array (EIA) from the 3D object is displayed. The flipped elemental image (EI)s are numerically formed by the virtual pinhole array. Then, the determined interval depending on the reconstruction plane is calculated and applied to each flipped EI. These flipped EIs are shifted to match the determined interval at the reconstruction plane and superimposed together. After this superimposed image is divided by the number of the superposition, the position error between the location of the shifted EI and the pixel position of the reconstruction plane is corrected by interpolation. As a result, the refocused image depending on the reconstruction plane can be reconstructed rapidly. From the experimental result, we confirmed that the proposed method largely decreased the computation time compared with the conventional method. In addition, we verified that the quality of the reconstruction by the proposed method is higher than the conventional method by the use of the structural similarity index method.

Integral imaging reconstruction system based on the human eye viewing mechanism.

For integral stereo imaging systems based on lens arrays, the cross-mixing of erroneous light rays between adjacent lenses seriously affects the quality of the reconstructed light field. In this paper, we proposed a light field reconstruction method based on the human eye viewing mechanism, which incorporates simplified human eye imaging into the integral imaging system. First, the light field model for specified viewpoint is established, and the distribution of the light source for each viewpoint is accurately calculated for the EIA generation algorithm of fixed viewpoint. Second, according to the ray tracing algorithm in this paper, non-overlapping EIA based on the human eye viewing mechanism is designed to suppress the amount of crosstalk rays fundamentally. The actual viewing clarity is improved with the same reconstructed resolution. Experimental results verify the effectiveness of the proposed method. The SSIM value is higher than 0.93, which verifies that the viewing angle range is increased to 62°.

Compression and reconstruction of photon counting integral imaging

集成成像技术作为一种重要的裸眼三维显示技术，在完整记录三维场景信息的同时，庞大的数据量给传输和存储带来了压力。为了实现图像的有效压缩和重构，根据光子计数集成成像的特点，基于分布式压缩感知理论，提出用于图像压缩与重构的方案。该方案将图像分为参考图像和非参考图像两类，对其设置不同的测量率并分别进行重构。为保证非参考图像的重构质量，提出一种联合重构算法。该算法首先对非参考图像进行分块测量，依据与参考图像之间的相关性进行图像块分类，然后结合参考图像测量值信息构建新的测量矢量，利用新的测量矢量完成初次图像重构。为了进一步提升图像重构质量，对初次重构结果进行二次残差补偿重构，获得最终重构结果。最后通过设置不同的测量率进行了大量实验，实验结果表明，所提算法在测量率为0.25时，图像重构质量可以达到30 dB，测量率为0.4时，图像质量可以达到35 dB，算法性能具有一定的优越性。

Three-Dimensional Integral Imaging with Enhanced Lateral and Longitudinal Resolutions Using Multiple Pickup Positions

In this paper, we propose an enhancement of three-dimensional (3D) image visualization techniques by using different pickup plane reconstructions. In conventional 3D visualization techniques, synthetic aperture integral imaging (SAII) and volumetric computational reconstruction (VCR) can be utilized. However, due to the lack of image information and shifting pixels, it may be difficult to obtain better lateral and longitudinal resolutions of 3D images. Thus, we propose a new elemental image acquisition and computational reconstruction to improve both the lateral and longitudinal resolutions of 3D objects. To prove the feasibility of our proposed method, we present the performance metrics, such as mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and peak-to-sidelobe ratio (PSR). Therefore, our method can improve both the lateral and longitudinal resolutions of 3D objects more than the conventional technique.

Information theoretic performance evaluation of 3D integral imaging.

Integral imaging (InIm) has proved useful for three-dimensional (3D) object sensing, visualization, and classification of partially occluded objects. This paper presents an information-theoretic approach for simulating and evaluating the integral imaging capture and reconstruction process. We utilize mutual information (MI) as a metric for evaluating the fidelity of the reconstructed 3D scene. Also we consider passive depth estimation using mutual information. We apply this formulation for optimal pitch estimation of integral-imaging capture and reconstruction to maximize the longitudinal resolution. The effect of partial occlusion in integral imaging 3D reconstruction using mutual information is evaluated. Computer simulation tests and experiments are presented.

3D image hiding using deep demosaicking and computational integral imaging

This paper proposes a 3D image hiding algorithm with deep learning based demosaicking and computational integral imaging methods. Compared with the previous color 3D image coding method, the proposed algorithm can greatly reduce the amount of ciphertext data by two-thirds through the color filter array (CFA). And the generated CFA is coded by our proposed block cellular automata coding method, which has high security. It is worth noting that The deep learning based demosaicking algorithm can effectively resolve the problem of low resolution that exists in the conventional integral imaging reconstruction algorithms.

Three-dimensional polarimetric integral imaging in photon-starved conditions: performance comparison between visible and long wave infrared imaging.

Three-dimensional (3D) polarimetric integral imaging (InIm) to extract the 3D polarimetric information of objects in photon-starved conditions is investigated using a low noise visible range camera and a long wave infrared (LWIR) range camera, and the performance between the two sensors is compared. Stokes polarization parameters and degree of polarization (DoP) are calculated to extract the polarimetric information of the 3D scene while integral imaging reconstruction provides depth information and improves the performance of low-light imaging tasks. An LWIR wire grid polarizer and a linear polarizer film are used as polarimetric objects for the LWIR range and visible range cameras, respectively. To account for a limited number of photons per pixel using the visible range camera in low light conditions, we apply a mathematical restoration model at each elemental image of visible camera to enhance the signal. We show that the low noise visible range camera may outperform the LWIR camera in detection of polarimetric objects under low illumination conditions. Our experiments indicate that for 3D polarimetric measurements under photon-starved conditions, visible range sensing may produce a signal-to-noise ratio (SNR) that is not lower than the LWIR range sensing. We derive the probability density function (PDF) of the 2D and 3D degree of polarization (DoP) images and show that the theoretical model demonstrates agreement to that of the experimentally obtained results. To the best of our knowledge, this is the first report comparing the polarimetric imaging performance between visible range and infrared (IR) range sensors under photon-starved conditions and the relevant statistical models of 3D polarimetric integral imaging.

Photon-counting 3D integral imaging with less than a single photon per pixel on average using a statistical model of the EM-CCD camera

We investigate photon-counting 3D integral imaging (PCII) with an electron multiplying charged-coupled device (EM-CCD) camera using dedicated statistical models. Using conventional integral imaging reconstruction methods with this camera in photon-counting conditions may result in degraded reconstructed image quality if multiple photons are detected simultaneously in a given pixel. We propose an estimation method derived from the photon detection statistical model of the EM-CCD to address the problems caused by multiple photons detected at the same pixel and provide improved 3D reconstructions. We also present a simplified version of this statistical method that can be used under the correct conditions. The imaging performance of these methods is evaluated on experimental data by the peak signal-to-noise ratio and the structural similarity index measure. The experiments demonstrate that 3D integral imaging substantially outperforms 2D imaging in degraded conditions. Furthermore, we achieve imaging in photon-counting conditions where, on average, less than a single photon per pixel is detected by the camera. To the best of our knowledge, this is the first report of PCII with the EM-CCD camera employing its statistical model in 3D reconstruction of PCII.

Optical 4D signal detection in turbid water by multi-dimensional integral imaging using spatially distributed and temporally encoded multiple light sources.

We propose an underwater optical signal detection system based on multi-dimensional integral imaging with spatially distributed multiple light sources and four-dimensional (4D) spatial-temporal correlation. We demonstrate our system for the detection of optical signals in turbid water. A 4D optical signal is generated from a three-dimensional (3D) spatial distribution of underwater light sources, which are temporally encoded using spread spectrum techniques. The optical signals are captured by an array of cameras, and 3D integral imaging reconstruction is performed, followed by multi-dimensional correlation to detect the optical signal. Inclusion of multiple light sources located at different depths allows for successful signal detection at turbidity levels not feasible using only a single light source. We consider the proposed system under varied turbidity levels using both Pseudorandom and Gold Codes for temporal signal coding. We also compare the effectiveness of the proposed underwater optical signal detection system to a similar system using only a single light source and compare between conventional and integral imaging-based signal detection. The underwater signal detection capabilities are measured through performance-based metrics such as receiver operating characteristic (ROC) curves, the area under the curve (AUC), and the number of detection errors. Furthermore, statistical analysis, including Kullback-Leibler divergence and Bhattacharya distance, shows improved performance of the proposed multi-source integral imaging underwater system. The proposed integral-imaging based approach is shown to significantly outperform conventional imaging-based methods.

Fourier focusing in integral imaging with optimum visualization pixels

In this paper, we propose Fourier Volumetric Computational Reconstruction (FVCR), which is a new computational integral imaging reconstruction (CIIR) algorithm. It handles physical pixel overlapping by refocusing in the Fourier domain. Thus, the longitudinal resolution of the reconstructed 3D image is the same as optical reconstruction. It also considers the lateral resolution limit of integral imaging and minimizes the computational load in CIIR. To verify the improvements in both lateral and longitudinal resolution of the reconstructed 3D image, in this paper, we implemented optical experiments. To the best of our knowledge, this is the first report of the efficient CIIR algorithm that considers the longitudinal and lateral resolution limits of integral imaging.

Copyright Protection for Holographic Video Using Spatiotemporal Consistent Embedding Strategy

The interest in a three-dimensional (3-D) holographic display is increasing daily because it can generate full depth cue without utilizing any special glasses. However, the content of 3-D holographic videos currently without any protection can be illegally distributed and maliciously manipulated. Therefore, there is an urgent need to protect the copyright of holographic videos against malicious copying. Since watermarking is an effective way to protect the ownership of holographic sequence, this paper proposes a spatiotemporal consistent embedding algorithm for the holographic video watermarking. Imperceptibility requirement in the holographic video watermarking is more challenging compared with static holograms because of the temporal dimension existing in videos. The embedding algorithm should not only consider spatially embedding strength for each frame of the video, but also take the temporal dimension into account in order to guarantee the visual quality of the moving object. Before embedding, to defend the imperceptibility of the watermark from the holographic moving object, the embedding parameters are evaluated by the salient object from the interframe and intraframe. Different from the previous video watermarking algorithm, in order to ensure the robustness, in our paper, instead of two-dimensional watermark, 3-D watermark converted data (QR code) are embedded in the cellular automata (CA) domains using 3-D CA filters. Finally, the QR codes can be extracted from the watermarked holographic frames, and the final 3-D watermark can be digitally reconstructed with different depths’ cue using the computational integral imaging reconstruction algorithm. The experimental results demonstrate that the proposed method exhibits superior performance compared to several methods in the literature, especially the robustness against additive noise and compression attacks.

3D view image reconstruction in computational integral imaging using scale invariant feature transform and patch matching.

This paper realizes a computational integral imaging reconstruction method via scale invariant feature transform (SIFT) and patch matching to improve the visual quality of reconstructed 3D view images. To our knowledge, the 3D view images reconstructed from the elemental images suffer from artifacts, which leads to degradations in the visual quality. To prevent image degradation, in this paper, we use the correct regions obtained from the view images taken directly from the original object or use patch matching to replace the distorted regions. However, the initial matching regions could not meet our requirements owing to the limitations of the equipment and the inevitable shortcomings of the experimental operation. To solve these problems, we adopt SIFT descriptors and perspective transform to get the satisfying correct regions. We present the simulation and experimental results of the 3D view images and the evaluation of the quality of the corresponding images to test the performance of the proposed method. The simulation and experimental results indicate that the proposed method can significantly improve the visual quality of the 3D view images and verify the feasibility and effectiveness of the proposed method.

Optical 3D object security and reconstruction using pixel-evaluated integral imaging algorithm.

Under the framework of computational integral imaging, an optical 3D objects security and high-quality reconstruction method based on pixel-evaluating mapping (PEM) algorithm is proposed. In this method, the pixel crosstalk caused by noneffective pixel overlap is effectively reduced by a pixel-evaluated mask, which can improve the image quality of the reconstructed 3D objects. Meanwhile, compared with the other computational integral imaging reconstruction methods, our proposed PEM algorithm can obtain more accurate pixel mapping weight parameters, thereby the reconstructed 3D objects provide higher quality. In addition, the nonlinear feedback shift register cellular automata algorithm is proposed to increase the security of the proposed method. We have experimentally verified the proposed 3D objects encryption and reconstruction algorithm. The experimental results show that the proposed method is superior to the other computational reconstruction methods.

Depth and All-in-Focus Image Estimation in Synthetic Aperture Integral Imaging Under Partial Occlusions

A common assumption in the integral imaging reconstruction is that a pixel will be photo-consistent if all viewpoints observed by the different cameras converge at a single point when focusing at the proper depth. However, the presence of occlusions between objects in the scene prevents this from being fulfilled. In this paper, a novel depth and all-in focus image estimation method is presented, based on a photo-consistency measure that uses the median criterion in relation to the elemental images. The interest of this approach is to find a solution to detect which camera correctly sees the partially occluded object at a certain depth and allows for a precise solution to the object depth. In addition, a robust solution is proposed to detect the boundary limits between partially occluded objects, which are subsequently used during the regularization depth estimation process. The experimental results show that the proposed method outperforms other state-of-the-art depth estimation methods in a synthetic aperture integral imaging framework.

Effective orthoscopic integral imaging reconstruction via adjustable depth position

In this paper, we propose an effective orthoscopic three-dimensional reconstruction via adjustable depth position in integral imaging system. Compared to conventional integral imaging reconstruction method, the proposed method can directly reconstruct orthoscopic 3D images through one-step ray tracing with adjustable depth positions. Thus, a real, undistorted, orthoscopic three-dimensional images can be reconstructed various distances without the magnification and overlap process of conventional integral imaging reconstruction. To show the feasibility of the proposed method, the preliminary experiments are carried out and the some experimental results are presented.

Modified integral imaging reconstruction and encryption using an improved SR reconstruction algorithm

We propose a monospectral image encryption method in which the multispectral color image acquisition by using heterogeneous monospectral cameras. Because the captured monospectral elemental images (EIs) belongs to grayscale image, it is means that the captured EIs can be directly encrypted by the proposed encoding method. Subsequently, the linear cellular automata (CA) and hyperchaotic encoding algorithm are employed to encrypt the captured EIs. Different from previous methods, the proposed method can directly encrypt the multispectral color information rather than having to divide into three color channels (R, G and B), thereby, the proposed method can greatly reduce the encryption calculation.