Synthetic Aperture Integral Imaging Research Articles

Three-Dimensional Integral Imaging with Enhanced Lateral and Longitudinal Resolutions Using Multiple Pickup Positions

In this paper, we propose an enhancement of three-dimensional (3D) image visualization techniques by using different pickup plane reconstructions. In conventional 3D visualization techniques, synthetic aperture integral imaging (SAII) and volumetric computational reconstruction (VCR) can be utilized. However, due to the lack of image information and shifting pixels, it may be difficult to obtain better lateral and longitudinal resolutions of 3D images. Thus, we propose a new elemental image acquisition and computational reconstruction to improve both the lateral and longitudinal resolutions of 3D objects. To prove the feasibility of our proposed method, we present the performance metrics, such as mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and peak-to-sidelobe ratio (PSR). Therefore, our method can improve both the lateral and longitudinal resolutions of 3D objects more than the conventional technique.

Motion-Blurring Free Integral Imaging Reconstruction via Parallax Information

In practical applications, the target objects or cameras are usually moving during the exposure time of the camera array in synthetic aperture integral imaging. Thus, the elemental images we captured are all blurred and can not be used to 3D reconstruction. The existing deblurring methods are most designed for single-image deblurring and just a few for multi-image deblurring. Nevertheless, all of these methods can not be used for our problem because they contain no parallax information. To address this problem, we proposed a motion-blurring free 3D reconstruction in synthetic aperture integral imaging with parallax information. We calculate all the PSFs by using reference PSF and parallax information to get a 3D reconstruction result rapidly. The results of experiments we performed show that our proposed method is effective in integral imaging motion deblurring and 3D reconstruction.

Depth and All-in-Focus Image Estimation in Synthetic Aperture Integral Imaging Under Partial Occlusions

A common assumption in the integral imaging reconstruction is that a pixel will be photo-consistent if all viewpoints observed by the different cameras converge at a single point when focusing at the proper depth. However, the presence of occlusions between objects in the scene prevents this from being fulfilled. In this paper, a novel depth and all-in focus image estimation method is presented, based on a photo-consistency measure that uses the median criterion in relation to the elemental images. The interest of this approach is to find a solution to detect which camera correctly sees the partially occluded object at a certain depth and allows for a precise solution to the object depth. In addition, a robust solution is proposed to detect the boundary limits between partially occluded objects, which are subsequently used during the regularization depth estimation process. The experimental results show that the proposed method outperforms other state-of-the-art depth estimation methods in a synthetic aperture integral imaging framework.

Synthetic aperture integral imaging using edge depth maps of unstructured monocular video.

Synthetic aperture integral imaging using monocular video with arbitrary camera trajectory enables casual acquisition of three-dimensional information of any-scale scenes. This paper presents a novel algorithm for computational reconstruction and imaging of the scenes in this SAII system. Since dense geometry recovery and virtual view rendering are required to handle such unstructured input, for less computational costs and artifacts in both stages, we assume flat surfaces in homogeneous areas and take full advantage of the per-frame edges which are accurately reconstructed beforehand. A dense depth map of each real view is first estimated by successively generating two complete, named smoothest- and densest-surface, depth maps, both respecting local cues, and then merging them via Markov random field global optimization. This way, high-quality perspective images of any virtual camera array can be synthesized simply by back-projecting the obtained closest surfaces into the new views. The pixel-level operations throughout most parts of our pipeline allow high parallelism. Simulation results have shown that the proposed approach is robust to view-dependent occlusions and lack of textures in original frames and can produce recognizable slice images at different depths.

Depth estimation improvement in 3D integral imaging using an edge removal approach

A new depth estimation method for 3D reconstruction in a synthetic aperture integral imaging framework is presented. This method removes the edges of the objects in the elemental images when the objects are in focus. This strategy aims to compensate for the noise that objects focused close to the cameras can introduce into the photo-consistency measure of objects at higher depths. Furthermore, a photo-consistency criterion is applied combining a defocus and a correspondence measure, and a depth regularization method which smooths noisy depth results for the case of object surfaces. The proposed method obtains consistent results for any type of object surfaces as well as very sharp boundaries. Experimental results show that our method reduces the noise in the object edges and gives rise to an improvement in the depth map results in relation to the other methods shown in the comparative analysis.

Designing optical 3D images encryption and reconstruction using monospectral synthetic aperture integral imaging.

This paper realizes an optical 3D images encryption and reconstruction by employing the geometric calibration algorithm to the monospectral synthetic aperture integral imaging system. This method has the simultaneous advantages of improving the quality of 3D images by eliminating the crosstalk from the unaligned cameras and increasing security of the multispectral 3D images encryption by importing the random generated maximum-length cellular automata into the Fresnel transform encoding algorithm. Furthermore, compared with the previous 3D images encryption methods of encrypting 3D multispectral information, the proposed method only encrypts monospectral data, which will greatly minimize the complexity. We present experimental results of 3D image encryption and volume pixel computational reconstruction to test and verify the performance of the proposed method. Experimental results validate the feasibility and robustness of our proposed approach, even under severe degradation.

Two-Step Integral Imaging Coding Based Three-Dimensional Information Encryption Approach

We propose a two-step integral imaging coding based three-dimensional (3D) information encryption approach in this paper. In this approach, a synthetic aperture integral imaging system is applied to acquire a set of parallax images including spatial and angular information of 3D scene. In the encryption process, two-step coding is performed. In the first step, the acquired parallax images are encrypted firstly by double random-phase coding in the Fresnel domain. In the second step, these encrypted parallax images are encoded into a cipher image by mapping algorithm which is used to generate elemental image array of integral imaging. In the decryption process, an inverse operation is performed. The experimental results demonstrate the feasibility, security, and robustness of the proposed approach.

Extended depth of field integral imaging using multi-focus fusion

In this paper, we propose a new method for depth of field extension in integral imaging by realizing the image fusion method on the multi-focus elemental images. In the proposed method, a camera is translated on a 2D grid to take multi-focus elemental images by sweeping the focus plane across the scene. Simply applying an image fusion method on the elemental images holding rich parallax information does not work effectively because registration accuracy of images is the prerequisite for image fusion. To solve this problem an elemental image generalization method is proposed. The aim of this generalization process is to geometrically align the objects in all elemental images so that the correct regions of multi-focus elemental images can be exacted. The all-in focus elemental images are then generated by fusing the generalized elemental images using the block based fusion method. The experimental results demonstrate that the depth of field of synthetic aperture integral imaging system has been extended by realizing the generation method combined with the image fusion on multi-focus elemental images in synthetic aperture integral imaging system.

Depth-of-field extension in integral imaging using multi-focus elemental images.

In integral imaging, one of the main challenges is the limited depth of field (DOF), which is mainly caused by the short focal length of the microlenses. In this paper, we propose a method to extend the DOF of a synthetic aperture integral imaging (SAII) system by realizing the image fusion method on the multi-focus elemental images with different perspectives. In the proposed system, the contour-based object extraction method combined with size correction is developed to solve size inconsistency of the objects in the misaligned elemental images. The all-in-focus elemental images (EIs) combining selected features of multi-focus elemental images are then obtained by the block-based image fusion method. In the last step, the reconstructed images with the extended DOF can be generated based on the all-in-focus EIs in the SAII system. Experimental results are presented to demonstrate the feasibility of the proposed system.

Compressive 4D spectro-volumetric imaging.

In this Letter, we present a method for hyperspectral imaging of three-dimensional objects. A compressive sensing approach is utilized to remedy the acquisition effort required to capture the large amount of data. The spectral dimension is compressively sensed by means of a liquid crystal-based encoder, and the volumetric data are captured using a synthetic aperture integral imaging setup. We demonstrate reconstruction of spectro-volumetric tesseracts with hundreds of spectral bands at different depths without compromise of spatial resolution.

Three-dimensional reconstruction of far and large objects using synthetic aperture integral imaging

In this paper, we present a three-dimensional reconstruction of far and large objects in a synthetic aperture integral imaging system. In the proposed method, the far and large size objects are recorded as a set of elemental images by using an additional Plano-concave lens in the synthetic aperture integral imaging system. Due to the use of the Plano-concave lens, the reconstruction distance can be significantly reduced. This enables us to computationally reconstruct the objects in the far-field region. Experimental results are carried out, and the feasibility of the proposed method is verified.

Multispectral high-resolution hologram generation using orthographic projection images

We present a new method of synthesizing a digital hologram of three-dimensional (3D) real-world objects from multiple orthographic projection images (OPI). A high-resolution multiple perspectives of 3D objects (i.e., two dimensional elemental image array) are captured under incoherent white light using synthetic aperture integral imaging (SAII) technique and their OPIs are obtained respectively. The reference beam is then multiplied with the corresponding OPI and integrated to form a Fourier hologram. Eventually, a modified phase retrieval algorithm (GS/HIO) is applied to reconstruct the hologram. The principle is validated experimentally and the results support the feasibility of the proposed method.

Depth estimation in Integral Imaging based on a maximum voting strategy

An approach that uses the scene information acquired by means of a three-dimensional (3D) synthetic aperture integral imaging system is presented. This method generates a depth map of the scene through a voting strategy. In particular, we consider the information given by each camera of the array for each pixel, and also the local information in the neighbourhood of that pixel. The proposed method obtains consistent results for any type of object surfaces as well as very sharp boundaries. In addition, we also contribute in this paper with a repository of a set of synthetic integral images generated by 3DS Max where the so-called ground truth (real-true depth map) is available. This resource can be used as a benchmark to test any Integral Imaging based range estimation method.

Depth Extraction of Partially Occluded 3D Objects Using Axially Distributed Stereo Image Sensing

There are several methods to record three dimensional (3D) information of objects such as lens array based integral imaging, synthetic aperture integral imaging (SAII), computer synthesized integral imaging (CSII), axially distributed image sensing (ADS), and axially distributed stereo image sensing (ADSS). ADSS method is capable of recording partially occluded 3D objects and reconstructing high-resolution slice plane images. In this paper, we present a computational method for depth extraction of partially occluded 3D objects using ADSS. In the proposed method, the high resolution elemental stereo image pairs are recorded by simply moving the stereo camera along the optical axis and the recorded elemental image pairs are used to reconstruct 3D slice images using the computational reconstruction algorithm. To extract depth information of partially occluded 3D object, we utilize the edge enhancement and simple block matching algorithm between two reconstructed slice image pair. To demonstrate the proposed method, we carry out the preliminary experiments and the results are presented.

Extended depth-of-field 3D endoscopy with synthetic aperture integral imaging using an electrically tunable focal-length liquid-crystal lens.

Conventional synthetic-aperture integral imaging uses a lens array to sense the three-dimensional (3D) object or scene that can then be reconstructed digitally or optically. However, integral imaging generally suffers from a fixed and limited range of depth of field (DOF). In this Letter, we experimentally demonstrate a 3D integral-imaging endoscopy with tunable DOF by using a single large-aperture focal-length-tunable liquid crystal (LC) lens. The proposed system can provide high spatial resolution and an extended DOF in synthetic-aperture integral imaging 3D endoscope. In our experiments, the image plane in the integral imaging pickup process can be tuned from 18 to 38 mm continuously using a large-aperture LC lens, and the total DOF is extended from 12 to 51 mm. To the best of our knowledge, this is the first report on synthetic aperture integral imaging 3D endoscopy with a large-aperture LC lens that can provide high spatial resolution 3D imaging with an extend DOF.

Fuzzy Integral Imaging Camera Calibration for Real Scale 3D Reconstructions

In this paper, we present a quantitative analysis of the error in the reconstruction of a 3D scene which has been captured with Synthetic Aperture Integral Imaging system. The 3D information is obtained from 2D images for which the camera parameters are unknown. The model used for calibrating the Integral Imaging camera setup is based on fuzzy systems. These systems provide the opportunity for modeling of conditions which are inherently imprecisely defined. We demonstrate that the error in the 3D reconstruction not only depends on the number of cameras, but also to their relative positions. Our model is applied to a set of images captured experimentally from a real object. A true-color real scale 3D reconstruction is successfully achieved.

Photoelastic Analysis of Partially Occluded Objects With an Integral-Imaging Polariscope

Polariscopes are the basic instruments used for the analysis of the stress state of transparent materials. Polarized light passing through a 3D object carries the integrated effect of the stress field along the light path. Therefore, conventional polariscopes are not able to discern the stress state of objects involving multiple plates with mutual occlusions. In this paper we propose a novel experimental system for three-dimensional stress analysis based on the combination of a polariscope and Synthetic Aperture Integral Imaging technique. Experimental results show the system's ability to recover the information of the stress distribution of a set of plates located at different depths having mutual occlusions.

Feasibility study for compressive multi-dimensional integral imaging

This paper describes a generalized framework for single-exposure acquisition of multi-dimensional scene information using integral imaging system based on compressive sensing. In the proposed system, a multi-dimensional scene containing a plurality of information such as 3D coordinates, spectral and polarimetric data is captured by integral imaging optics. The image sensor uses pixel-wise filtering elements arranged randomly. The multi-dimensional original object is reconstructed using an algorithm with a sparsity constraint. The proposed system is demonstrated with simulations and feasible optical experiments based on synthetic aperture integral imaging using multi-dimensional objects including 3D coordinates, spectral, and polarimetric information.

Multispectral integral imaging acquisition and processing using a monochrome camera and a liquid crystal tunable filter

This paper presents an acquisition system and a procedure to capture 3D scenes in different spectral bands. The acquisition system is formed by a monochrome camera, and a Liquid Crystal Tunable Filter (LCTF) that allows to acquire images at different spectral bands in the [480, 680]nm wavelength interval. The Synthetic Aperture Integral Imaging acquisition technique is used to obtain the elemental images for each wavelength. These elemental images are used to computationally obtain the reconstruction planes of the 3D scene at different depth planes. The 3D profile of the acquired scene is also obtained using a minimization of the variance of the contribution of the elemental images at each image pixel. Experimental results show the viability to recover the 3D multispectral information of the scene. Integration of 3D and multispectral information could have important benefits in different areas, including skin cancer detection, remote sensing and pattern recognition, among others.

Synthetic aperture 집적 영상을 이용한 3D 영상 디스플레이 방법

Synthetic aperture 집적 영상 (SAII) 기술은 다수의 카메라를 이용하여 고해상도의 요소 영상을 획득할 수 있는 유망한 3D 이미징 기술이다. 본 논문에서는 SAII 기술을 이용하여 집적 영상 디스플레이를 수행하는 공간 3D 영상을 표시하는 방법을 제안한다. SAII로부터 얻어지는 요소 영상은 직접적으로 공간 3D 영상으로 사용될 수 없기 때문에 깊이 지도를 추출하여 새로운 디스플레이용 요소 영상으로 변환하여 공간 3D 영상을 표시한다. 제안하는 방법의 유용함을 보이기 위해서 장난감 3D 물체를 사용하여 기초적인 실험을 수행하고, 또한 공간 3D 영상이 구현된 실험 결과를 제시한다. Synthetic aperture integral imaging is one of promising 3D imaging techniques to capture the high-resolution elemental images using multiple cameras. In this paper, we propose a method of displaying 3D images in space using the synthetic aperture integral imaging technique. Since the elemental images captured from SAII cannot be directly used to display 3D images in an integral imaging display system, we first extract the depth map from elemental images and then transform them to novel elemental images for 3D image display. The newly generated elemental images are displayed on a display panel to generate 3D images in space. To show the usefulness of the proposed method, we carry out the preliminary experiments using a 3D toy object and present the experimental results.