Three-dimensional Microscopic Imaging Research Articles

A high temporal-spatial resolution three-dimensional imaging method using arrayed-focus structured illumination with differential axial enhancement

High precision and fast three-dimensional (3D) microscopic imaging technology plays an extremely important role in MEMS, micro optics, microelectronics, biomedicine and other fields. In this paper, a confocal microscopic 3D imaging method based on differential array focusing structured light illumination (DAF-SIM) is proposed. Theoretical analysis and experimental results show that under the 100x/NA 0.95 objective lens, the lateral resolution of this method reaches 173 nm, exceeding the optical diffraction limit, the axial sensitivity is better than 10 nm, and the time resolution is about 3 Hz@2.3 M Pixel, achieving the goal of high horizontal and axial resolution and time resolution. The high spatial and temporal resolution of DAF-SIM 3D micro imaging method provides a highly applicable, efficient and high-precision online micro terrain detection scheme for intelligent manufacturing and other fields.

C1M2: a universal algorithm for 3D instance segmentation, annotation, and quantification of irregular cells.

Cell instance segmentation is a fundamental task for many biological applications, especially for packed cells in three-dimensional (3D) microscope images that can fully display cellular morphology. Image processing algorithms based on neural networks and feature engineering have enabled great progress in two-dimensional (2D) instance segmentation. However, current methods cannot achieve high segmentation accuracy for irregular cells in 3D images. In this study, we introduce a universal, morphology-based 3D instance segmentation algorithm called Crop Once Merge Twice (C1M2), which can segment cells from a wide range of image types and does not require nucleus images. C1M2 can be extended to quantify the fluorescence intensity of fluorescent proteins and antibodies and automatically annotate their expression levels in individual cells. Our results suggest that C1M2 can serve as a tissue cytometry for 3D histopathological assays by quantifying fluorescence intensity with spatial localization and morphological information.

Metalens-based stereoscopic microscope

Stereoscopic microscopy is a promising technology to obtain three-dimensional microscopic images. Such microscopes are based on the parallax effect, and as such require two lenses to focus at two different points. Geometrical constraints, however, restrict their numerical apertures to about 0.2, thus limiting the system’s resolution. Higher numerical apertures ( ∼ 0.35 ) can be achieved with designs using only one bulk lens, but such systems are ∼ 10 times more costly than the conventional ones. Thus, there is a pressing need for alternative solutions to improve the resolution of stereoscopic systems. Here, we show that high-resolution and low-cost stereoscopic systems can be obtained using birefringent single-layer metalenses. We design and fabricate a birefringent metalens operating at 532 nm with a numerical aperture as high as 0.4. The metalens is then used to demonstrate high-resolution stereoscopic imaging of biological samples. The microscopic images are further displayed and perceived vividly in an autostereoscopic display. Our demonstration paves the way to a new strategy to achieve high-resolution and low-cost stereoscopic microscopes.

Three-dimensional microscopic single-pixel imaging with chaotic light

Three-dimensional (3D) microscopic imaging is fundamentally important for microscale observations and manipulations. Here, we show that single-pixel imaging (SPI) is available for realizing superior 3D microscopic imaging, which relies on the near-field illumination with chaotic light. Like considering a two-point object to investigate two-dimensional microscopic imaging, we theoretically and experimentally studied the image formation process for a 3D object with two slices, demonstrating the excellent capability of the SPI for 3D microscopic imaging. The proposed 3D microscopic imaging via SPI would be useful for applications such as biological imaging and micromanipulations.

Three-Dimensional Microscopic Image Reconstruction Based on Structured Light Illumination.

In this paper, we propose and experimentally demonstrate a three-dimensional (3D) microscopic system that reconstructs a 3D image based on structured light illumination. The spatial pattern of the structured light changes according to the profile of the object, and by measuring the change, a 3D image of the object is reconstructed. The structured light is generated with a digital micro-mirror device (DMD), which controls the structured light pattern to change in a kHz rate and enables the system to record the 3D information in real time. The working distance of the imaging system is 9 cm at a resolution of 20 μm. The resolution, working distance, and real-time 3D imaging enable the system to be applied in bridge and road crack examinations, and structure fault detection of transportation infrastructures.

Testing System for the Mechanical Properties of Small-Scale Specimens Based on 3D Microscopic Digital Image Correlation.

The testing of the mechanical properties of materials on a small scale is difficult because of the small specimen size and the difficulty of measuring the full-field strain. To tackle this problem, a testing system for investigating the mechanical properties of small-scale specimens based on the three-dimensional (3D) microscopic digital image correlation (DIC) combined with a micro tensile machine is proposed. Firstly, the testing system is described in detail, including the design of the micro tensile machine and the 3D microscopic DIC method. Then, the effects of different shape functions on the matching accuracy obtained by the inverse compositional Gauss–Newton (IC-GN) algorithm are investigated and the numerical experiment results verify that the error due to under matched shape functions is far larger than that of overmatched shape functions. The reprojection error is shown to be smaller than before when employing the modified iteratively weighted radial alignment constraint method. Both displacement and uniaxial measurements were performed to demonstrate the 3D microscopic DIC method and the testing system built. The experimental results confirm that the testing system built can accurately measure the full-field strain and mechanical properties of small-scale specimens.

Micrometer-resolution X-ray tomographic full-volume reconstruction of an intact post-mortem juvenile rat lung

In this article, we present an X-ray tomographic imaging method that is well suited for pulmonary disease studies in animal models to resolve the full pathway from gas intake to gas exchange. Current state-of-the-art synchrotron-based tomographic phase-contrast imaging methods allow for three-dimensional microscopic imaging data to be acquired non-destructively in scan times of the order of seconds with good soft tissue contrast. However, when studying multi-scale hierarchically structured objects, such as the mammalian lung, the overall sample size typically exceeds the field of view illuminated by the X-rays in a single scan and the necessity for achieving a high spatial resolution conflicts with the need to image the whole sample. Several image stitching and calibration techniques to achieve extended high-resolution fields of view have been reported, but those approaches tend to fail when imaging non-stable samples, thus precluding tomographic measurements of large biological samples, which are prone to degradation and motion during extended scan times. In this work, we demonstrate a full-volume three-dimensional reconstruction of an intact rat lung under immediate post-mortem conditions and at an isotropic voxel size of (2.75 µm)3. We present the methodology for collecting multiple local tomographies with 360° extended field of view scans followed by locally non-rigid volumetric stitching. Applied to the lung, it allows to resolve the entire pulmonary structure from the trachea down to the parenchyma in a single dataset. The complete dataset is available online (https://doi.org/10.16907/7eb141d3-11f1-47a6-9d0e-76f8832ed1b2).

Digital holographic microscopy for red blood cell imaging

Digital holographic microscopy (DHM) can obtain biological parameters and morphological information of cells by reconstructing holograms, which is different from traditional optical microscopy. The DHM is a three-dimensional imaging technology which is effective, non-contact and non-destructive. With the developments of the image sensor and the computing technology, it has made significant progress in the field of living cells detection, especially for red blood cell. Compared with the technologies which are widely used in the field of cell imaging such as con-focal laser scanning microscopy, scanning near-field optical microscopy and optical coherence tomography, the DHM has the advantages including wide FOV and high-resolution to achieve higher imaging and quality. This paper introduces the principle of recording and reconstruction of digital holography, and then analyzes the performance of three reconstruction algorithms using the Fresnel method, the convolution method and the angular spectrum method. The Fresnel method is suitable for the sample size larger than the image sensor. Both the convolution method and the angular spectrum method have an optimal reconstruction distance. When the reconstruction distance is different from the optimal distance, the resolution of the reconstructed image will decrease, and the angular spectrum method is better than the convolution method in overall performance. The DHM system for RBC measurements mainly adopts the convolution algorithm or the angular spectrum algorithm to implement numerical reconstruction. The systems of the in-line DHM, the off-axis DHM and the optical tweezers combining with off-axis DHM are introduced. These techniques use algorithms including Rayleigh-Sommerfeld back-propagation, the sharpness quantification, the watershed segmentation, the numerical refocusing and the thermal fluctuation to determine the focal plane position and obtain the best reconstruction distance of the RBCs, and further detect the shape change of the RBCs and extract the information of high-resolution blood vessel shape and blood flow velocity. These techniques can even achieve the dynamic tracking and measure three-dimensional volume of RBCs in real-time which is helpful for pathological studies such as diabetes, cardiovascular disease and Parkinson's disease. With its unique non-contact and non-destructive characteristics, the DHM realizes real-time and quantitative detection that is difficult to achieve with traditional three-dimensional microscopic imaging technologies.

Calibration of a stereo microscope based on non-coplanar feature points with iteratively weighted radial alignment constraint

A commercial stereo light microscope, such as the OLYMPUS SZX7, has nonnegligible lens distortion for three-dimensional microscopic image correlation. To tackle the problem, an effective calibration method of using a stereo-vision system is proposed. It is implemented using non-coplanar feature points with weighted radial constraints, where the weights are incorporated into the objective function to estimate the internal and external parameters of the vision system iteratively. In the calculation of the distortion coefficients, a non-parametric distortion model expressed by a bicubic spline surface is used. The iterative convergence of the objective function and the influence of the attitude number of the calibration target on the calibration results are analysed, and verified with a digital stereo light microscope established with OLYMPUS SZX7. Both shape and displacement measurements have been performed to demonstrate the proposed method. The experimental results show that the proposed method can achieve higher accuracy compared with Zhang's classical method and the unweighted two-step method.

Three-dimensional microscopic imaging through scattering media based on in-line phase-shift digital holography.

Microscopic three-dimensional imaging and phase quantification for objects hidden behind a scattering medium by using in-line phase-shift digital holography are proposed. A spatial resolution of 1.81 µm and highly accurate quantitative phase imaging are demonstrated for objects behind a scatter plate. Three-dimensional imaging was confirmed using objects with a depth difference of 1.32 mm. Further, imaging was performed using rat skin as a demonstration for imaging through a complex multilayer scattering medium, where a spatial resolution close to the theoretically predicted value was achieved by experiment.

Dual wavelength digital holographic imaging of layered structures

We present a dual wavelength digital holographic technique for three-dimensional microscopic imaging of layered structures, where layers are separated from one another by the axial distances exceeding the wavelength of imaging light. Our methodology not only provides the three-dimensional structure of each layer, but also allows the height differentiation of distinct layers. We have also implemented a technique suppressing low intensity signal when no reliable phase information can be extracted, based on the quality of the interference fringe pattern. We utilize a dual wavelength setup, where the combination of two overlapping interferometers enables simultaneous acquisition of two phase profiles. We demonstrate that this imaging modality is particularly well-suited for imaging of multilayered electrode structures embedded in glass.

Investigation of the Tensile Fracture Behavior of Notched C/SiC Composite Using <i>In Situ</i> SEM and Micro-CT

In order to better understand the failure mechanism of C/SiC composites, the tensile behavior of notched C/SiC composites was investigated by the in-situ scanning electron microscopy (SEM) and the micro-CT technique. Surface morphologies of the C/SiC sample during tensile loading were in-situ observed by SEM, while the three-dimensional microscopic images of the C/SiC sample before loading and after failure were obtained by micro-CT. The results showed that no cracks formed in the initial elastic stage corresponding to the linear part of the load-displacement curve. However, corresponding to the following non-linear part of the load-displacement curve, matrix crack initiation, fiber pull-out, crack propagation and deflection appeared consecutively in the notched region of the sample. What’s more, different crack growth paths existed in different directions of the sample during tensile failure. In general, approximately flat fracture formed in the plying direction and serrated or stepped fracture were observed in the needling direction. It indicated that the in-situ observation method combining SEM and micro-CT can obtain the micro-structure images of the material in different states, which is helpful to analyze the fracture failure mechanism of composites.

Synthetic Bessel light needle for extended depth-of-field microscopy.

An ultra-long light needle is highly desired in optical microscopy for its ability to improve the lateral resolution over a large depth of field (DOF). However, its use in image acquisition usually relies on mechanical raster scanning, which compromises between imaging speed and stability and thereby restricts imaging performance. Here, we propose a synthetic Bessel light needle (SBLN) that can be generated and scanned digitally by complex field modulation using a digital micromirror device. In particular, the SBLN achieves a 45-fold improvement in DOF over its counterpart Gaussian focus. Further, we apply the SBLN to perform motionless two-dimensional and three-dimensional microscopic imaging, achieving both improved resolution and extended DOF. Our work is expected to open up opportunities for potential biomedical applications.

Three-dimensional microscopic light field particle image velocimetry

A microscopic particle image velocimetry ( $$\mu \text {PIV}$$ ) technique is developed based on light field microscopy and is applied to flow through a microchannel containing a backward-facing step. The only hardware difference from a conventional $$\mu$$ PIV setup is the placement of a microlens array at the intermediate image plane of the microscope. The method combines this optical hardware alteration with post-capture computation to enable 3D reconstruction of particle fields. From these particle fields, we measure three-component velocity fields, but find that accurate velocity measurements are limited to the two in-plane components at discrete depths through the volume (i.e., 2C-3D). Results are compared with a computational fluid dynamics simulation.

High-definition optical coherence tomography imaging for noninvasive examination of heritage works

Cultural heritage works, such as ancient murals and historical paintings, are examined routinely for the purpose of conservation. Previous works have applied optical coherence tomography (OCT), which is a three-dimensional (3D) microscopic imaging modality in the field of heritage works conservation. The data acquired by OCT provides both 3D surface information of the object and structure information underneath the surface. Therefore, cross-sectional information on the object can be utilized to study layer structure of the painting and brush stroke techniques used by the artist. However, as demonstrated in previous studies, OCT has limited capability in high-definition (HD) examination of paintings or murals that are in macroscopic scale. HD examination of heritage works needs to scan large areas and process huge amounts of data, while OCT imaging has a limited field of view and processing power. To further advance the application of OCT in the conservation of heritage works, we demonstrate what we believe is a novel high-speed, large field-of-view (FOV) OCT imaging platform. Our results suggest that this OCT platform has the potential to become a nondestructive alternative for the analysis and conservation of paintings and murals.

Interactive visual exploration of overlapping similar structures for three-dimensional microscope images.

BackgroundRecent advances in microscopy enable the acquisition of large numbers of tomographic images from living tissues. Three-dimensional microscope images are often displayed with volume rendering by adjusting the transfer functions. However, because the emissions from fluorescent materials and the optical properties based on point spread functions affect the imaging results, the intensity value can differ locally, even in the same structure. Further, images obtained from brain tissues contain a variety of neural structures such as dendrites and axons with complex crossings and overlapping linear structures. In these cases, the transfer functions previously used fail to optimize image generation, making it difficult to explore the connectivity of these tissues.ResultsThis paper proposes an interactive visual exploration method by which the transfer functions are modified locally and interactively based on multidimensional features in the images. A direct editing interface is also provided to specify both the target region and structures with characteristic features, where all manual operations can be performed on the rendered image. This method is demonstrated using two-photon microscope images acquired from living mice, and is shown to be an effective method for interactive visual exploration of overlapping similar structures.ConclusionsAn interactive visualization method was introduced for local improvement of visualization by volume rendering in two-photon microscope images containing regions in which linear nerve structures crisscross in a complex manner. The proposed method is characterized by the localized multidimensional transfer function and interface where the parameters can be determined by the user to suit their particular visualization requirements.

New deconvolution method for microscopic images based on the continuous Gaussian radial basis function interpolation model.

A deconvolution method based on the Gaussian radial basis function (GRBF) interpolation is proposed. Both the original image and Gaussian point spread function are expressed as the same continuous GRBF model, thus image degradation is simplified as convolution of two continuous Gaussian functions, and image deconvolution is converted to calculate the weighted coefficients of two-dimensional control points. Compared with Wiener filter and Lucy-Richardson algorithm, the GRBF method has an obvious advantage in the quality of restored images. In order to overcome such a defect of long-time computing, the method of graphic processing unit multithreading or increasing space interval of control points is adopted, respectively, to speed up the implementation of GRBF method. The experiments show that based on the continuous GRBF model, the image deconvolution can be efficiently implemented by the method, which also has a considerable reference value for the study of three-dimensional microscopic image deconvolution.

A generic classification-based method for segmentation of nuclei in 3D images of early embryos

BackgroundStudying how individual cells spatially and temporally organize within the embryo is a fundamental issue in modern developmental biology to better understand the first stages of embryogenesis. In order to perform high-throughput analyses in three-dimensional microscopic images, it is essential to be able to automatically segment, classify and track cell nuclei. Many 3D/4D segmentation and tracking algorithms have been reported in the literature. Most of them are specific to particular models or acquisition systems and often require the fine tuning of parameters.ResultsWe present a new automatic algorithm to segment and simultaneously classify cell nuclei in 3D/4D images. Segmentation relies on training samples that are interactively provided by the user and on an iterative thresholding process. This algorithm can correctly segment nuclei even when they are touching, and remains effective under temporal and spatial intensity variations. The segmentation is coupled to a classification of nuclei according to cell cycle phases, allowing biologists to quantify the effect of genetic perturbations and drug treatments. Robust 3D geometrical shape descriptors are used as training features for classification. Segmentation and classification results of three complete datasets are presented. In our working dataset of the Caenorhabditis elegans embryo, only 21 nuclei out of 3,585 were not detected, the overall F-score for segmentation reached 0.99, and more than 95% of the nuclei were classified in the correct cell cycle phase. No merging of nuclei was found.ConclusionWe developed a novel generic algorithm for segmentation and classification in 3D images. The method, referred to as Adaptive Generic Iterative Thresholding Algorithm (AGITA), is freely available as an ImageJ plug-in.

Visualization and analysis of 3D microscopic images.

In a wide range of biological studies, it is highly desirable to visualize and analyze three-dimensional (3D) microscopic images. In this primer, we first introduce several major methods for visualizing typical 3D images and related multi-scale, multi-time-point, multi-color data sets. Then, we discuss three key categories of image analysis tasks, namely segmentation, registration, and annotation. We demonstrate how to pipeline these visualization and analysis modules using examples of profiling the single-cell gene-expression of C. elegans and constructing a map of stereotyped neurite tracts in a fruit fly brain.

Soft x-ray tomoholography

We demonstrate an x-ray imaging method that combines Fourier transform holography with tomography (‘tomoholography’) for three-dimensional (3D) microscopic imaging. A 3D image of a diatom shell with a spatial resolution of 140 nm is presented. The experiment is realized by using a small gold sphere as the reference wave source for holographic imaging. This setup allows us to rotate the sample and to collect a number of 2D projections for tomography.