Elemental Image Array Research Articles

P‐101: Method of Accelerated Elemental Image Array Generation for Integral Imaging Display

In this paper, we propose an accelerated elemental image array (EIA) generation method for integral imaging display. The EIA is directly generated by the calculated mapping matrices which are from original parallaxes to the EIA. We simplify the camera array calibration process, parallax alignment and interweaving processes into a simple remapping operation. The experimental results show that the proposed method can significantly accelerate the EIA generation.

Tabletop augmented reality 3D display system based on integral imaging

In this paper, we propose a tabletop augmented reality (AR) three-dimensional (3D) display system based on integral imaging by using a holographic optical element (HOE) that performs the optical function of a microlens array. The microlens array HOE records the wavefronts of a set of tilted spherical waves based on the theory of reflective volume holograms. In the reconstruction, an elemental image array projected by a projector and collimated by relay optics satisfies Bragg matching conditions and reconstructs the tilted spherical waves. Based on the theory of integral imaging, a 3D image is generated by the tilting spherical waves in an oblique viewing angle, which presents the tabletop 3D feature. Meanwhile, because of the wavelength and angular selectivity of the reflective volume hologram, the Bragg mismatched lights emitted from the real-world scene directly pass through the microlens array HOE. The 3D image and the real-world scene are seamlessly combined, which presents the AR feature. The experimental results confirm that the proposed system can provide a 3D virtual image in an oblique viewing angle and the real-world scene behind the system can also be clearly presented.

Real object-based integral imaging system using a depth camera and a polygon model

An integral imaging system using a polygon model for a real object is proposed. After depth and color data of the real object are acquired by a depth camera, the grid of the polygon model is converted from the initially reconstructed point cloud model. The elemental image array is generated from the polygon model and directly reconstructed. The polygon model eliminates the failed picking area between the points of a point cloud model, so at least the quality of the reconstructed 3-D image is significantly improved. The theory is verified experimentally, and higher-quality images are obtained.

Designing a genetic watermarking approach for 3D scenes

A new watermarking algorithm based on genetic algorithm (GA) in the transform domain is proposed. Unlike the existing computer-generated integral imaging based watermarking methods, the proposed method utilizes GA searching to the optimized transform domain to serve as a trade-off for watermark embedding. In this paper, 3D scene to be captured by using a virtual pinhole array and be computationally recorded as an elemental image array (EIA), watermarking with GA optimization and computer-generated holography is implemented. In the proposed GA optimization process, we utilize the fitness function to improve the visual quality of watermarked images and the robustness. Simulation results show that the proposed algorithm yields a holographic watermark that is imperceptibility to human eyes and robust to standard watermarking attacks. A comparison of the proposed watermarking method to the existing similar watermarking methods demonstrated that the proposed method generally outperforms completing methods in terms of imperceptibility and robustness.

High-efficient computer-generated integral imaging based on the backward ray-tracing technique and optical reconstruction.

A high-efficient computer-generated integral imaging (CGII) method is presented based on the backward ray-tracing technique. In traditional CGII methods, the total rendering time is long, because a large number of cameras are established in the virtual world. The ray origin and the ray direction for every pixel in elemental image array are calculated with the backward ray-tracing technique, and the total rendering time can be noticeably reduced. The method is suitable to create high quality integral image without the pseudoscopic problem. Real time and non-real time CGII rendering images and optical reconstruction are demonstrated, and the effectiveness is verified with different types of 3D object models. Real time optical reconstruction with 90 × 90 viewpoints and the frame rate above 40 fps for the CGII 3D display are realized without the pseudoscopic problem.

Visual perception based robust watermarking with integral imaging

This paper presents a robust watermarking approach for hiding gray watermark into digital images. Conventional transform-based watermarking schemes provide only one transform plane for watermark embedding, which makes easy to remove embedded watermark information. Cellular automata transform (CAT) based watermarking scheme can address this problem with CA properties. In this paper, the watermark is first recorded in the form of an elemental image array (EIA) with the computational integral imaging (CII) system, and then the recorded EIA is embedded into the CAT coefficient. The watermark embedding coefficient will be determined by an alpha map which generated by saliency map. This proposed method can provide a more robust watermarking system because EIA is composed of many elemental images and each image has its own property of the watermark. Even though most data of elemental images are lost, the watermark can be reconstructed successfully by using the remaining elemental images. Furthermore, the alpha map which is generated by saliency map can make watermark more invisible for the human visual system. Some experimental results show competence of our system.

Multispectral high-resolution hologram generation using orthographic projection images

We present a new method of synthesizing a digital hologram of three-dimensional (3D) real-world objects from multiple orthographic projection images (OPI). A high-resolution multiple perspectives of 3D objects (i.e., two dimensional elemental image array) are captured under incoherent white light using synthetic aperture integral imaging (SAII) technique and their OPIs are obtained respectively. The reference beam is then multiplied with the corresponding OPI and integrated to form a Fourier hologram. Eventually, a modified phase retrieval algorithm (GS/HIO) is applied to reconstruct the hologram. The principle is validated experimentally and the results support the feasibility of the proposed method.

Encrypting 2D/3D image using improved lensless integral imaging in Fresnel domain

We propose a new image encryption technique, for the first time to our knowledge, combined Fresnel transform with the improved lensless integral imaging technique. In this work, before image encryption, the input image is first recorded into an elemental image array (EIA) by using the improved lensless integral imaging technique. The recorded EIA is encrypted into random noise by use of two phase masks located in the Fresnel domain. The positions of phase masks and operation wavelength, as well as the integral imaging system parameters are used as encryption keys that can ensure security. Compared with previous works, the main novelty of this proposed method resides in the fact that the elemental images possess distributed memory characteristic, which greatly improved the robustness of the image encryption algorithm. Meanwhile, the proposed pixel averaging algorithm can effectively address the overlapping problem existing in the computational integral imaging reconstruction process. Numerical simulations are presented to demonstrate the feasibility and effectiveness of the proposed method. Results also indicate the high robustness against data loss attacks.

Resolution Enhanced Computational Integral Imaging Reconstruction by Using Boundary Folding Mirrors

In this paper, we present a resolution-enhanced computational integral imaging reconstruction method by using boundary folding mirrors. In the proposed method, to improve the resolution of the computationally reconstructed 3D images, the direct and reflected light information of the 3D objects through a lenslet array with boundary folding mirrors is recorded as a combined elemental image array. Then, the ray tracing method is employed to synthesize the regular elemental image array by using a combined elemental image array. From the experimental results, we can verify that the proposed method can improve the visual quality of the computationally reconstructed 3D images.

Three-dimensional integral imaging display system via off-axially distributed image sensing

In this paper, we propose a three-dimensional integral imaging display system with a multiple recorded images using off-axially distributed image sensing. First, the depth map of the 3D objects is extracted from the off-axially recorded multi-perspective 2D images by using profilometry technique. Then, the elemental image array is computationally synthesized using the extracted depth map based on ray mapping model. Finally, the 3D images are optically displayed in integral imaging system. To show the feasibility of the proposed method, the optical experiments for 3D objects are carried out and presented in this paper.

Encrypting three-dimensional information system based on integral imaging and multiple chaotic maps

An encrypting three-dimensional (3-D) information system based on integral imaging (II) and multiple chaotic maps is proposed. In the encrypting process, the elemental image array (EIA) which represents spatial and angular information of the real 3-D scene is picked up by a microlens array. Subsequently, R, G, and B color components decomposed by the EIA are encrypted using multiple chaotic maps. Finally, these three encrypted components are interwoven to obtain the cipher information. The decryption process implements the reverse operation of the encryption process for retrieving the high-quality 3-D images. Since the encrypted EIA has the data redundancy property due to II, and all parameters of the pickup part are the secret keys of the encrypting system, the system sensitivity on the changes of the plaintext and secret keys can be significantly improved. Moreover, the algorithm based on multiple chaotic maps can effectively enhance the security. A preliminary experiment is carried out, and the experimental results verify the effectiveness, robustness, and security of the proposed system.

Integral imaging microscopy with enhanced depth-of-field using a spatial multiplexing

A depth-of-field enhancement method for integral imaging microscopy system using a spatial multiplexing structure consisting of a beamsplitter with dual video channels and micro lens arrays is proposed. A computational integral imaging reconstruction algorithm generates two sets of depth-sliced images for the acquired depth information of the captured elemental image arrays and the well-focused depth-slices of both image sets are combined where each is focused on a different depth plane of the specimen. A prototype is implemented, and the experimental results demonstrate that the depth-of-field of the reconstructed images in the proposed integral imaging microscopy is significantly increased compared with conventional integral imaging microscopy systems.

Dual-view integral imaging 3D display by using orthogonal polarizer array and polarization switcher.

In this paper, a dual-view integral imaging three-dimensional (3D) display consisting of a display panel, two orthogonal polarizer arrays, a polarization switcher, and a micro-lens array is proposed. Two elemental image arrays for two different 3D images are presented by the display panel alternately, and the polarization switcher controls the polarization direction of the light rays synchronously. The two elemental image arrays are modulated by their corresponding and neighboring micro-lenses of the micro-lens array, and reconstruct two different 3D images in viewing zones 1 and 2, respectively. A prototype of the dual-view II 3D display is developed, and it has good performances.

Chaotic image encryption using pseudo-random masks and pixel mapping

Integral imaging-based cryptographic algorithms provide a new way to design secure and robust image encryption systems. In this paper, we introduce a performance-enhanced image encryption scheme based on depth-conversion integral imaging and hybrid cellular automata (CA), aiming to meet the requirements of secure image transmission. First, the input image is decomposed into an elemental image array (EIA) using the depth-converted integral imaging technique. The obtained elemental images then are encrypted by utilizing the CA model and chaotic sequence. The conventional computational integral imaging reconstruction (CIIR) technique is a pixel-superposition technique. The resolution of the reconstructed image is dramatically degraded by the large magnification factor in the superposition process as the pickup distance increases. In the proposed reconstruction process, the pixel mapping technique is introduced to solve these problems. A novel property of the proposed scheme is its depth-conversion property, which reconstructs an elemental image originally recorded at long distances from the pinhole array as one that was recorded near the pinhole array and consequently reduces the magnification factor. The results of numerical simulations demonstrate the effectiveness and security of the proposed scheme.

Integral imaging display for natural scene based on KinectFusion

In this paper, we propose an integral imaging display system based on KinectFusion, using only a moving low-cost depth camera as the pickup device. The multi-frame depth data of the observed scene streamed from a Kinect sensor is fused into a single global surface model represented by the volumetric truncated signed distance function. Thus, the inherent noise associated with single frame depth data can be eliminated. The elemental image array for display is obtained by ray casting the volumetric truncated signed distance function. To match the pickup part and the display part, the relationship between the pickup voxel and the display voxel is deduced using the ray optics theory. Experimental results show the validity of the proposed method.

Resolution-enhanced integral imaging using two micro-lens arrays with different focal lengths for capturing and display

We proposed a resolution enhanced integral imaging display method using two micro-lens arrays (MLA) with different focal lengths for capturing and display respectively. An elemental image array (EIA) is captured with MLA of focal length of f(1) and a processed EIA is displayed with MLA of focal length of f(2) which is larger than f(1). We enlarge the "effective area" in processed EIA to increase the information obtained by viewer, in other words, enhance the viewing resolution. The two micro-lens arrays for capturing and display are g and mg distant from display device respectively, and we can get m(2) times resolution enhancement.

Active integral imaging system based on multiple structured light method.

In this paper, we simplify the equipments for integral imaging (II) pickup and implement an active II system based on multiple structured light (MSL) method. In the active II system, the complete three-dimensional (3D) shape of the 3D scene can be reconstructed, and the tunable parallaxes can be generated without occlusions. Therefore, the high-quality 3D images can be displayed efficiently by the II. We also use the Compute Unified Device Architecture implementing the processing algorithms in graphics processing unit. The experimental results demonstrate the effectiveness of the MSL method for the II pickup and the acceleration for the elemental image array generation. Especially, the proposed method is suitable for the real scene with high precision.

Statistics-based reconstruction method with high random-error tolerance for integral imaging.

A three-dimensional (3D) digital reconstruction method for integral imaging with high random-error tolerance based on statistics is proposed. By statistically analyzing the points reconstructed by triangulation from all corresponding image points in an elemental images array, 3D reconstruction with high random-error tolerance could be realized. To simulate the impacts of random errors, random offsets with different error levels are added to a different number of elemental images in simulation and optical experiments. The results of simulation and optical experiments showed that the proposed statistic-based reconstruction method has relatively stable and better reconstruction accuracy than the conventional reconstruction method. It can be verified that the proposed method can effectively reduce the impacts of random errors on 3D reconstruction of integral imaging. This method is simple and very helpful to the development of integral imaging technology.

A tri-view integral imaging three-dimensional display

We propose a tri-view integral imaging (TVII) three-dimensional (3D) display which presents three different 3D images in the left, middle and right viewing directions simultaneously. The element image array (EIA) of the TVII 3D display consists of three sub-EIAs. The three sub-EIAs captured from three different scenes are responsible for three different 3D images in the left-, middle- and right-view, respectively. A prototype of the TVII display is developed, and the experimental results verify the correctness of the theory. The TVII 3D display is easily convertible to mono-view or dual-view II 3D display.

An innovative calibration based integral photography rendering algorithm for medical application and its evaluation.

Autostereoscopic has long been proposed to fulfill medical display in image-guided surgery and clinical education to provide more intuitive position information of clinical interest zone thus improving surgery safety and accuracy. As one category of flexible autostereoscopic 3D display, computer generated integral photography (CGIP) has been studied in medical application by many researches for its convenience and cost-efficiency. However, IP still suffers from inaccurate light field reconstruction, which limits its practicality in surgery. In this paper, we propose and apply a flexible fish-eye model based micro lens array (MLA) distortion calibration method and pre-distorted retracing rendering algorithm to render elemental image array (EIA) of CGIP. Furthermore, we also evaluate light field of the proposed algorithm in depth cue, and signal noise ratio of IP images by phantom experiment.