Integral Photography Images Research Articles

Measurement of accommodation response in integral photography images

A method of displaying a multiview image set prepared by a computer on an electronic display device based on integral photography has been formulated. With the device, accommodation response was measured and the result that the accommodation response linearly changed within and outside the range of depth of focus was obtained. In contrast, in the measurement of the accommodation response using the stereoscopic image, the accommodation response showed a linear change within and outside the depth of focus at the spatial resolution of three cycle per degree (cpd) or less. Therefore, it can be considered that the measurement result of the accommodation response for integral photography with a spatial resolution of 3 cpd or less cannot be used to determine whether it is influenced by the integral photography display method or spatial resolution reduction. From these results, we found that spatial resolution is required to be 3 cpd or more when outside of the range of depth of focus in the measurement of the accommodation response for integral photography.

Measurement of static convergence and accommodation responses to images of integral photography and binocular stereoscopy.

Static convergence and accommodation responses were measured by comparing integral photography images, binocular stereoscopic images, and real objects in a measurement range from 450 to 900 mm. The experimental results were evaluated with a multiple comparison test. It was found that six of the ten observers did not have an accommodation-convergence conflict in viewing integral photography in the range. Moreover, the required resolution was found to be 0.7 or more and less than 1.4 cycles per degree for inducing accommodation. In conclusion, integral photography can provide a natural 3D image that looks like a real object.

360 degree viewable floating autostereoscopic display using integral photography and multiple semitransparent mirrors.

In this paper, we present a polyhedron-shaped floating autostereoscopic display viewable from 360 degrees using integral photography (IP) and multiple semitransparent mirrors. IP combined with polyhedron-shaped multiple semitransparent mirrors is used to achieve a 360 degree viewable floating three-dimensional (3D) autostereoscopic display, having the advantage of being able to be viewed by several observers from various viewpoints simultaneously. IP is adopted to generate a 3D autostereoscopic image with full parallax property. Multiple semitransparent mirrors reflect corresponding IP images, and the reflected IP images are situated around the center of the polyhedron-shaped display device for producing the floating display. The spatial reflected IP images reconstruct a floating autostereoscopic image viewable from 360 degrees. We manufactured two prototypes for producing such displays and performed two sets of experiments to evaluate the feasibility of the method described above. The results of our experiments showed that our approach can achieve a floating autostereoscopic display viewable from surrounding area. Moreover, it is shown the proposed method is feasible to facilitate the continuous viewpoint of a whole 360 degree display without flipping.

Real-time capture and reconstruction system with multiple GPUs for a 3D live scene by a generation from 4K IP images to 8K holograms

We developed a real-time capture and reconstruction system for three-dimensional (3D) live scenes. In previous research, we used integral photography (IP) to capture 3D images and then generated holograms from the IP images to implement a real-time reconstruction system. In this paper, we use a 4K (3,840 × 2,160) camera to capture IP images and 8K (7,680 × 4,320) liquid crystal display (LCD) panels for the reconstruction of holograms. We investigate two methods for enlarging the 4K images that were captured by integral photography to 8K images. One of the methods increases the number of pixels of each elemental image. The other increases the number of elemental images. In addition, we developed a personal computer (PC) cluster system with graphics processing units (GPUs) for the enlargement of IP images and the generation of holograms from the IP images using fast Fourier transform (FFT). We used the Compute Unified Device Architecture (CUDA) as the development environment for the GPUs. The Fast Fourier transform is performed using the CUFFT (CUDA FFT) library. As a result, we developed an integrated system for performing all processing from the capture to the reconstruction of 3D images by using these components and successfully used this system to reconstruct a 3D live scene at 12 frames per second.

Characteristics of composite images in multiview imaging and integral photography

The compositions of images projected to a viewer's eyes from the various viewing regions of the viewing zone formed in one-dimensional integral photography (IP) and multiview imaging (MV) are identified. These compositions indicate that they are made up of pieces from different view images. Comparisons of the composite images with images composited at various regions of imaging space formed by camera arrays for multiview image acquisition reveal that the composite images do not involve any scene folding in the central viewing zone for either MV or IP. However, in the IP case, compositions from neighboring viewing regions aligned in the horizontal direction have reversed disparities, but in the viewing regions between the central and side viewing zones, no reversed disparities are expected. However, MV does exhibit them.

Integral Photography Using 2D Printer Output and Fly's Eye Lens Made with 3D Printer

A new integral photography (IP) technique is presented in which printing technology is used to make not only an IP image but also a fly's eye lens. IP is one of the best methods for 3D displays because both horizontal and vertical parallax can be obtained without having to wear special glasses. A fly's eye lens, which integrates a large number of small convex lenses in array form, is the key component of IP. However, such lenses had to be purchased from a limited number of commercially available options since using a metal mold to custom make them was extremely costly. This situation changed substantially with the emergence of a 3D printer that could produce highly precise and transparent lenses easily and inexpensively. We therefore used our original software to model a fly's eye lens that comprised many minute spherical lenses as a triangular mesh. The resulting STereoLithography (STL) file was then transferred to a 3D printer and a fly's eye lens was made. We also used our conventional CG technology to synthesize an IP image that integrates images observed from many viewpoints. When the image was printed with a full-color inkjet printer and the fly's eye lens was put on it, an excellent 3D image was obtained.

Autostereoscopic 3D Display with Long Visualization Depth Using Referential Viewing Area-Based Integral Photography

We developed an autostereoscopic display for distant viewing of 3D computer graphics (CG) images without using special viewing glasses or tracking devices. The images are created by employing referential viewing area-based CG image generation and pixel distribution algorithm for integral photography (IP) and integral videography (IV) imaging. CG image rendering is used to generate IP/IV elemental images. The images can be viewed from each viewpoint within a referential viewing area and the elemental images are reconstructed from rendered CG images by pixel redistribution and compensation method. The elemental images are projected onto a screen that is placed at the same referential viewing distance from the lens array as in the image rendering. Photographic film is used to record the elemental images through each lens. The method enables 3D images with a long visualization depth to be viewed from relatively long distances without any apparent influence from deviated or distorted lenses in the array. We succeeded in creating an actual autostereoscopic images with an image depth of several meters in front of and behind the display that appear to have 3D even when viewed from a distance.

TransCAIP: A Live 3D TV System Using a Camera Array and an Integral Photography Display with Interactive Control of Viewing Parameters

The system described in this paper provides a real-time 3D visual experience by using an array of 64 video cameras and an integral photography display with 60 viewing directions. The live 3D scene in front of the camera array is reproduced by the full-color, full-parallax autostereoscopic display with interactive control of viewing parameters. The main technical challenge is fast and flexible conversion of the data from the 64 multicamera images to the integral photography format. Based on image-based rendering techniques, our conversion method first renders 60 novel images corresponding to the viewing directions of the display, and then arranges the rendered pixels to produce an integral photography image. For real-time processing on a single PC, all the conversion processes are implemented on a GPU with GPGPU techniques. The conversion method also allows a user to interactively control viewing parameters of the displayed image for reproducing the dynamic 3D scene with desirable parameters. This control is performed as a software process, without reconfiguring the hardware system, by changing the rendering parameters such as the convergence point of the rendering cameras and the interval between the viewpoints of the rendering cameras.

Integral color rainbow hologram based on computer image processing

A new method for obtaining color rainbow holograms of 3D scenes using single wavelength laser is proposed and implemented by combining integral photography, computer graphics and rainbow holography. In the method, three integral photography (IP) images whose colors correspond to three primary colors are calculated by simulating integral photography, and exported on the film. By illuminating the IP images with diffused coherent beam from the opposite side, three images of the 3D scene are reproduced and overlapped on a particular image plane by the three linear micro-lens arrays. The linear micro-lens arrays can be regarded as the slits. A color rainbow hologram can be created after a single exposure by interfering the reproduced 3D images and the reference beam. Furthermore, we demonstrate experimentally the new method for a simple case and the results can be generalized in integral photography.

Thin and Flexible Integral Photography using High-Resolution Printer Output

We present a new 3D printing technology in which a thin and flexible fly's eye lens sheet and high-resolution printer output are used. Integral photography (IP) is one of the best stereoscopic viewing systems available because not only horizontal but also vertical parallax can be obtained without the need to wear special glasses. As previous fly's eye lenses were thick and hard, they were quite unlike paper. We used a flexible fly's eye lens sheet made from polypropylene with a thickness of only 0.5 mm. The density of small convex lenses is 80 lines per inch (lpi). A high-resolution printer is needed to extract images that demonstrate the full performance of this minute lens. However, recent progress in printer technology means that even a common ink-jet printer meets this demand. Two kinds of lens sheets were used; one consisted of square lenses, and the other consisted of hexagonal ones. We rendered images of 3D objects made with CG from 1024 different angles, and used our software to synthesize an IP image from them. The IP image was printed with an ink-jet printer, and observed through one of the lens sheets mentioned above. We observed good 3D images that exhibited the full parallax effect.

Hardware implementation of a disparity estimation scheme for real-time compression in 3D imaging applications

This paper presents a novel hardware implementation of a disparity estimation scheme targeted to real-time Integral Photography (IP) image and video sequence compression. The software developed for IP image compression achieves high quality ratios over classic methodologies by exploiting the inherent redundancy that is present in IP images. However, there are certain time constraints to the software approach that must be confronted in order to address real-time applications. Our main effort is to achieve real-time performance by implementing in hardware the most time-consuming parts of the compression algorithm. The proposed novel digital architecture features minimized memory read operations and extensive simultaneous processing, while taking into concern the memory and data bandwidth limitations of a single FPGA implementation. Our results demonstrate that the implemented hardware system can successfully process high resolution IP video sequences in real-time, addressing a vast range of applications, from mobile systems to demanding desktop displays.

Enhanced reconstruction of three-dimensional shape and texture from integral photography images

A method for the reconstruction of 3D shape and texture from integral photography (IP) images is presented. Sharing the same principles with stereoscopic-based object reconstruction, it offers increased robustness to noise and occlusions due to the unique characteristics of IP images. A coarse-to-fine approach is used, employing what we believe to be a novel grid refinement step in order to increase the quality of the reconstructed objects. The proposed method's properties include configurable depth accuracy and direct and seamless triangulation. We evaluate our method using synthetic data from a computer-simulated IP setup as well as real data from a simple yet effective digital IP setup. Experiments show reconstructed objects of high-quality indicating that IP can be a competitive modality for 3D object reconstruction.

Image Synthesis Method for Integral Photography Using Hexagonal Fly's Eye Lens

A new method of synthesizing an image is presented, which is essential for integral photography, using high-resolution printer output and a fly's eye lens sheet. Integral photography (IP) is one of the best stereoscopic viewing systems available because not only horizontal but also full parallax can be obtained without the need for wearing special glasses. A special image, called an IP image, is synthesized with a computer using this technology, and is output with a high-resolution printer. When the image is observed through a special lens sheet, called a fly's eye lens, superb binocular vision can be obtained. The lens sheet consists of an array of minute convex lenses. Each lens is either square or hexagonal and the latter has the advantage that the number of lenses for each unit area can be increased. However, the existing method of synthesis is no longer applicable. Therefore, we propose a method of synthesis that can be applied to these cases. Half the proposed method is the same as the existing one in which hundreds of still images are taken with a real or virtual camera from different angles. However, the next stage of the method is only for cases using the hexagonal fly's eye lens. The displacement from the center of the nearest hexagonal lens is calculated for all pixels of in the IP image. Since the displacement corresponds to the direction of a ray emitted from a pixel, the nearest camera position to the ray is calculated from hundreds of camera positions. A pixel position that corresponds to the hexagonal lens is calculated from the image taken from the camera position, and the pixel value is set to the pixel of the IP image. Finally, the IP image is scaled to the final IP image so that the size matches the hexagonal fly's eye lens sheet. Experiments demonstrated that excellent binocular vision could be obtained using the new method.

Photorealistic integral photography using a ray-traced model of capturing optics

We present a new approach for computer-generated integral photography (IP) based on ray tracing, for the reconstruction of high quality photorealistic 3-D images of increased complexity. With the proposed methodology, all the optical elements of a single-stage IP capturing setup are physically modeled for the production of real and virtual orthoscopic IP images with depth control. This approach is straightforward for translating a computer-generated 3-D scene to an IP image, and constitutes a robust methodology for developing modules that can be easily integrated in existing ray tracers. An extension of this technique enables the generation of photorealistic 3-D videos [integral videography (IV)] and provides an invaluable tool for the development of 3-D video processing algorithms.

Generation of Integral Photography Images Using Light Wave 3D™

Integral photography (IP) is an ideal 3D image display system because observers can see realistic 3D images from arbitrary directions without wearing special glasses. In addition, IP is excellent since both pinhole arrays and IP images can be manufactured by printing special patterns on transparent sheets. This paper proposes a new method of synthesizing an IP image by using LightWave 3D, which is commercially available 3D software commonly used worldwide. A series of still images, each of which is taken from a different camera position are required. In order to automate it, the animation function of the software is utilized. Namely, a camera position corresponds to a key frame, and all the key frames are generated at a time by using an LScript, which is a script language developed for LightWave 3D. Under such a condition, still images taken from different camera positions can be acquired as sequentially numbered bmp images. The still images are synthesized by our original software to get an IP image. By experiments it was confirmed that proper 3D images can be observed with the proposed method.

Synthesis of Integral Photography Images Using Shade™

This paper proposes a new technique to produce high quality integral photography (IP) images. IP is an ideal 3D image display system in which viewers can see realistic 3D images from arbitrary directions without wearing special glasses. In addition, IP is excellent with printing technologies since both pinhole arrays and IP images can be printed on transparent sheets by high resolution printers. Up to now IP images were produced by using CG or photography, but the quality of the CG images was not always satisfactory, since only simple objects such as cones or spheres were used. Shade™ is a commercial three dimensional CG application with abundant powers of expression, and is very popular among professional CG artists especially in Japan, since it is suitable for expressing complex shapes such as human bodies with delicacy. By controlling the camera position of Shade by a script language, and synthesizing an IP image from resultant still images according to our algorithm, the authors achieved the 3D display of realistic scenes created with Shade.

3D Image Input System for High Resolution Integral Photography

Integral photography (IP), in which viewers can see 3D images from almost any viewing points without wearing special glasses, is one of the ideal 3D display technologies. Generally an IP image, which is composed of many element images, is captured through a lens array. However, this method is not suited to produce high resolution 3D images, because only small number of picture elements is assigned to each element image. We propose a 3D image input system for high resolution IP. In this system a CCD camera is attached to an XY plotter instead of a pen, and 32 × 32=1024 images are successively captured by the camera as it moves vertically and horizontally. The captured images are processed by our program to compose an IP image. We examined the IP image by using it in our 3D display system, which consists of two transparent films, a transparent board and illumination. The IP image is printed in advance on one of the transparent films by high resolution ink jet printer, and pinholes are printed on the other transparent film. By experiments, it has been confirmed that composite IP images generated by our system reproduce proper 3D images.

3-D computer graphics based on integral photography

Integral photography (IP), which is one of the ideal 3-D photographic technologies, can be regarded as a method of capturing and displaying light rays passing through a plane. The NHK Science and Technical Research Laboratories have developed a real-time IP system using an HDTV camera and an optical fiber array. In this paper, the authors propose a method of synthesizing arbitrary views from IP images captured by the HDTV camera. This is a kind of image-based rendering system, founded on the 4-D data space Representation of light rays. Experimental results show the potential to improve the quality of images rendered by computer graphics techniques.

Real-time pickup method for a three-dimensional image based on integral photography

We studied integral photography (IP), which creates three-dimensional autostereoscopic images. In particular we studied the possibility of a new method that uses a television camera to shoot directly numerous real images produced by a lens array. Unlike the conventional IP method in which the film is placed immediately behind a lens array, this method employs a television camera, which enables us to shoot moving pictures. Of a number of factors affecting the process of image pickup, we examined some optical factors and compared them with those obtained by the conventional IP method. The results show that with this new direct pickup method that uses a television camera, we can obtain an IP image like those obtained by using the conventional IP method. Further, we conducted an experiment with an high-definition TV camera, confirming the production of an autostereoscopic image by using a display device that combines a liquid-crystal panel and pinholes.