Integral Imaging Microscopy Research Articles

High-quality 3D display system for an integral imaging microscope using a simplified direction-inversed computation based on user interaction.

We propose and implement a high-quality three-dimensional (3D) display system for an integral imaging microscope using a simplified direction-inversed computation method based on user interaction. A model of the specimen is generated from the estimated depth information (via the convolutional neural network-based algorithm), the quality of the model is defined by the high-resolution two-dimensional image. The new elemental image arrays are generated from the models via a simplified direction-inversed computation method according to the user interaction and directly displayed on the display device. A high-quality 3D visualization of the specimen is reconstructed and displayed while the lens array is placed in front of the display device. The user interaction enables more viewpoints of the specimen to be reconstructed by the proposed system, within the basic viewing zone. Remarkable quality improvement is confirmed through quantitative evaluations of the experimental results.

Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy.

The integral imaging microscopy system provides a three-dimensional visualization of a microscopic object. However, it has a low-resolution problem due to the fundamental limitation of the F-number (the aperture stops) by using micro lens array (MLA) and a poor illumination environment. In this paper, a generative adversarial network (GAN)-based super-resolution algorithm is proposed to enhance the resolution where the directional view image is directly fed as input. In a GAN network, the generator regresses the high-resolution output from the low-resolution input image, whereas the discriminator distinguishes between the original and generated image. In the generator part, we use consecutive residual blocks with the content loss to retrieve the photo-realistic original image. It can restore the edges and enhance the resolution by ×2, ×4, and even ×8 times without seriously hampering the image quality. The model is tested with a variety of low-resolution microscopic sample images and successfully generates high-resolution directional view images with better illumination. The quantitative analysis shows that the proposed model performs better for microscopic images than the existing algorithms.

Advanced Three-Dimensional Visualization System for an Integral Imaging Microscope Using a Fully Convolutional Depth Estimation Network

In this paper, we propose an advanced three-dimensional visualization method for an integral imaging microscope system to simultaneously improve the resolution and quality of the reconstructed image. The main advance of the proposed method is that it generates a high-quality three-dimensional model without limitation of resolution by combining the high-resolution two-dimensional color image with depth data obtained through a fully convolutional neural network. First, the high-resolution two-dimensional image and an elemental image array for a specimen are captured, and the orthographic-view image is reconstructed from the elemental image array. Then, via a convolutional neural network-based depth estimation after the brightness of input images are uniformed, a more accurate and improved depth image is generated; and the noise of result depth image is filtered. Subsequently, the estimated depth data is combined with the high-resolution two-dimensional image and transformed into a high-quality three-dimensional model. In the experiment, it was confirmed that the displayed high-quality three-dimensional model could be visualized very similarly to the original image.

Resolution-Enhancement for an Integral Imaging Microscopy Using Deep Learning

A novel resolution-enhancement method for an integral imaging microscopy that applies interpolation and deep learning is proposed, and the complete system with both hardware and software components is implemented. The resolution of the captured elemental image array is increased by generating intermediate-view elemental images between each neighboring elemental image, and an orthographic-view visualization of the specimen is reconstructed. Then, a deep learning algorithm is used to generate maximum possible resolution for each reconstructed directional-view image with improved quality. Since a pretrained model is applied, the proposed system processes the images directly without data training. The experimental results indicate that the proposed system produces resolution-enhanced directional-view images, and quantitative evaluation methods for reconstructed images such as the peak signal-to-noise ratio and the power spectral density confirm that the proposed system provides improvements in image quality.

Optical sectioning with a Wiener-like filter in Fourier integral imaging microscopy

Non-scanning, single-shot, 3D integral microscopy with optical sectioning is presented. The method is based on the combination of Fourier-mode integral microscopy with a 3D deconvolution technique. Specifically, the refocused volume provided by a regular back-projection algorithm is 3D deconvolved with a synthetic 3D impulse response function that takes into account the number and positions of the elemental images. The use of this hybrid technique provides a stack of true-color depth-refocused images with significant gain of optical sectioning. The stack can be used, among other applications, to inspect inside the thick microscope specimen, to calculate collections of perspective views with fine angular resolution and extended full parallax, and also to display 3D images in an integral monitor. The method here presented is validated with both simulation and experimental data.

Optimized stereo matching algorithm for integral imaging microscopy and its potential use in precise 3-D optical manipulation

Abstract In this paper, we present an integral imaging microscopy, which can achieve continuous depth extraction. The multiple stereo matching algorithm is optimized with data point interpolation and calculation range correction, which can improve the computing efficiency and accuracy. A water suspension of silica-embedded magnetic particles in light-trap environment is used as a specimen. The elementary image array is obtained through sequentially imaging the specimen by the objective lens and microlens array, which contains continuous depth information that can give the optical tweezers a feedback for precise 3-D manipulation. Experimental and calculated results show the feasibility of our method.

Magnification enhancement for an integral imaging microscope system

A novel integral imaging microscope system with high magnification is analyzed and implemented in this paper. In our system, we use an inverted fluorescence microscope for specimens amplification, and apply a 3-D information acquisition system for 3-D information acquisition, and utilize an EMCCD for acquiring elemental images arrays formed by a microlens array and relay system. By using our experimental prototype, we record 12 × 12 elemental images arrays of different type specimens successfully, and generate depth slice images by CIIR method. Comparing with a conventional light field microscope, the magnification of our system is improved hundreds times because of remaining eyepiece and designing novel experimental configuration, then we can acquire more fine 3-D information of the specimen. Moreover, we develop a diffuse transmission element for acquiring 3-D information which is carried by scattered light, so we can acquire more plentiful 3-D information. Finally, we also demonstrate the validity and the effectiveness of our high magnification system by biological microorganism experimental data.

Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array.

An integral imaging microscopy (IIM) system with improved depth-of-field (DoF) using a custom-designed bifocal polarization-dependent liquid-crystalline polymer micro lens array (LCP-MLA) is proposed. The implemented MLA has improved electro-optical properties such as a small focal ratio, high fill factor, low driving voltage, and fast switching speed, utilizing a well-aligned reactive mesogen on the imprinted reverse shape of the lens and a polarization switching layer. A bifocal MLA switches its focal length according to the polarization angle and acquires different DoF information of the specimen. After two elemental image arrays are captured, the depth-slices are reconstructed and combined to provide a widened DoF. The fabricated bifocal MLA consists of two identical polarization-dependent LCP-MLAs with 1.6 mm and f/16 focal ratio. Our experimental results confirmed that the proposed system improves the DoF of IIM without the need for mechanical manipulation.

Enhanced depth-of-field of an integral imaging microscope using a bifocal holographic optical element-micro lens array

We propose and implement an integral imaging microscope with extended depth-of-field (DoF) using a bifocal holographic micro lens array (MLA). The properties of the two MLAs are switched via peristrophic multiplexing, where different properties of the MLA are recorded onto the single holographic optical element (HOE). The recorded MLA properties are perpendicular to each other: after the first mode is recorded, the HOE is rotated by 90° clockwise, and the second mode is recorded. The experimental results confirm that the DoF of the integral imaging microscopy system is extended successfully by using the bifocal MLA.

Integral imaging microscopy with enhanced depth-of-field using a spatial multiplexing

A depth-of-field enhancement method for integral imaging microscopy system using a spatial multiplexing structure consisting of a beamsplitter with dual video channels and micro lens arrays is proposed. A computational integral imaging reconstruction algorithm generates two sets of depth-sliced images for the acquired depth information of the captured elemental image arrays and the well-focused depth-slices of both image sets are combined where each is focused on a different depth plane of the specimen. A prototype is implemented, and the experimental results demonstrate that the depth-of-field of the reconstructed images in the proposed integral imaging microscopy is significantly increased compared with conventional integral imaging microscopy systems.

광자 계수 집적 영상 현미경을 사용한 마이크로 물체의 3차원 시각화와 인식

광자 계수 집적 영상 현미경을 사용한 마이크로 물체의 3차원 시각화와 인식

Resolution-enhancement for an orthographic-view image display in an integral imaging microscope system.

Due to the limitations of micro lens arrays and camera sensors, images on display devices through the integral imaging microscope systems have been suffering for a low-resolution. In this paper, a resolution-enhanced orthographic-view image display method for integral imaging microscopy is proposed and demonstrated. Iterative intermediate-view reconstructions are performed based on bilinear interpolation using neighborhood elemental image information, and a graphics processing unit parallel processing algorithm is applied for fast image processing. The proposed method is verified experimentally and the effective results are presented in this paper.

Real-time interactive display for integral imaging microscopy.

A real-time interactive orthographic-view image display of integral imaging (II) microscopy that includes the generation of intermediate-view elemental images (IVEIs) for resolution enhancement is proposed. Unlike the conventional II microscopes, parallel processing through a graphics processing unit is required for real-time display that generates the IVEIs and interactive orthographic-view images in high speed, according to the user interactive input. The real-time directional-view display for the specimen for which 3D information is acquired through II microscopy is successfully demonstrated by using resolution-enhanced elemental image arrays. A user interactive feature is also satisfied in the proposed real-time interactive display for II microscopy.

Analysis on enhanced depth of field for integral imaging microscope

Depth of field of the integral imaging microscope is studied. In the integral imaging microscope, 3-D information is encoded as a form of elemental images Distance between intermediate plane and object point decides the number of elemental image and depth of field of integral imaging microscope. From the analysis, it is found that depth of field of the reconstructed depth plane image by computational integral imaging reconstruction is longer than depth of field of optical microscope. From analyzed relationship, experiment using integral imaging microscopy and conventional microscopy is also performed to confirm enhanced depth of field of integral imaging microscopy.

Resolution-enhanced integral imaging microscopy that uses lens array shifting

A resolution-enhanced integral imaging microscope that uses lens array shifting is proposed in this study. The lens shift method maintains the same field of view of the reconstructed orthographic view images with increased spatial density. In this study, multiple sets of the elemental images were captured with horizontal and vertical shifts of the micro lens array and combined together to form a single set of the elemental images. From the combined elemental images, orthographic view images and depth slice images of the microscopic specimen were generated with enhanced resolution.