Single-shot 3D Research Articles

Single-shot 3D shape measurement based on RGB dot patterns and stereovision.

One-shot projection structured light 3D measurement is a method to establish the stereo matching relationship and reconstruct 3D shape by projecting one pattern. However, the traditional stereo matching algorithm does not solve the problem of low matching accuracy and matching efficiency, which fundamentally limits the accuracy of 3D measurement. As the projector and imaging systems have daily higher resolution and imaging quality, RGB dots projection has more application prospects because of its ability to establish a stereo matching relationship through one projection. In this work, we proposed a single-shot 3D measurement method using line clustering stereo matching, and model correction methods. The projected RGB dots are extracted by slope differenced distribution and area constrained erosion method. Area constrained erosion can solve the problem of the segmented connected blobs caused by insufficient projection resolution. The clustering stereo matching method is utilized to coarse match the segmented center red points. A model correction method is utilized to restore and constrain the pattern that cannot be imaged. Experimental results demonstrated that our method achieves the best accuracy of about 0.089mm, better than the traditional disparity and RGB line method, which may shed light on the proposed method can accurately reconstruct the 3D surface.

Deep learning-based Phase Measuring Deflectometry for single-shot 3D shape measurement and defect detection of specular objects.

Phase Measuring Deflectometry (PMD) and Structured-Light Modulation Analysis Technique (SMAT) perform effectively in shape and defect measurements of specular objects, but the difficulty of giving consideration to accuracy and speed has also restricted the further development and application of them. Inspired by recent successes of deep learning techniques for computational imaging, we demonstrate for the first time that deep learning techniques can be used to recover high-precision modulation distributions of specular surfaces from a single-frame fringe pattern under SMAT, enabling fast and high-quality defect detection of specular surfaces. This method can also be applied to recover higher-precision phase distributions of specular surfaces from a single-frame fringe pattern under PMD, so as to realize the 3D shape measurement. In this paper, we combine depthwise separable convolution, residual structure and U-Net to build an improved U-Net network. The experimental results prove that the method has excellent performance in the phase and modulation retrieval of specular surfaces, which almost reach the accuracy of the results obtained by ten-step phase-shifting method.

Single-Shot 3D Topography of Transmissive and Reflective Samples with a Dual-Mode Telecentric-Based Digital Holographic Microscope.

Common path DHM systems are the most robust DHM systems as they are based on self-interference and are thus less prone to external fluctuations. A common issue amongst these DHM systems is that the two replicas of the sample’s information overlay due to self-interference, making them only suitable for imaging sparse samples. This overlay has restricted the use of common-path DHM systems in material science. The overlay can be overcome by limiting the sample’s field of view to occupy only half of the imaging field of view or by using an optical spatial filter. In this work, we have implemented optical spatial filtering in a common-path DHM system using a Fresnel biprism. We have analyzed the optimal pinhole size by evaluating the frequency content of the reconstructed phase images of a star target. We have also measured the accuracy of the system and the sensitivity to noise for different pinhole sizes. Finally, we have proposed the first dual-mode common-path DHM system using a Fresnel biprism. The performance of the dual-model DHM system has been evaluated experimentally using transmissive and reflective microscopic samples.

Single-shot 3D tracking based on polarization multiplexed Fourier-phase camera: erratum

This erratum corrects Fig. 5 in Photon. Res. 9, 1924 (2021)PRHEIZ2327-912510.1364/PRJ.432292.

Single-shot high-precision 3D reconstruction with color fringe projection profilometry based BP neural network

In order to reduce the influence of nonlinear factors such as gamma nonlinearity and color crosstalk on the phase retrieval accuracy with color fringe projection profilometry (CFPP) in rapid measurement, a single-shot high-precision backpropagation (BP) neural network based CFPP is proposed. Firstly, neural networks are trained from samples through a training process. The neural network inputs are the RGB components of composite color fringe patterns. The numerator and denominator of the arctangent function are used as network outputs. The monochromatic fringe patterns are adopted. The numerator and denominator of the arctangent function are calculated by the 15-step phase-shift method, which serves as the network training target. Applying this neural network to the prediction of the standard phase can reduce the influence of nonlinear factors on the phase accuracy. Then, the absolute phase was unwrapped by using geometric constraints. Finally, the 3D reconstruction with high precision can be realized by using only a single-shot color fringe. The theoretical model of the proposed method and the process of neural network training are introduced. The results of 3D reconstruction are compared with other algorithms in the same condition. The experimental results demonstrate that the RMS of the proposed algorithm can reach 0.0107 mm, compared with 0.0121 mm of the traditional 3-step phase-shift method. The proposed method is applied to the online quality inspection of relay components, and the results verify the high accuracy and high efficiency of measurement.

Single-shot N-step phase measurement profilometry

A single-shot N-step phase measurement profilometry (single-shot N-PMP) is proposed. In the traditional N-step PMP, N (N > 2) frames of phase-shifting sinusoidal gratings are needed to be projected onto the measured object. The corresponding N frames of phase-shifting deformed patterns modulated by the object are needed to be captured. And the larger the N is, the higher the measuring accuracy will be. In the proposed method, only one sinusoidal grating is needed to be projected and only one corresponding deformed pattern is captured. The proposed method can efficiently avoid direct filtering in the spectrum of the deformed pattern because of the secondary modulation in computer. The proposed method is an innovative single-shot, three-dimensional (3D) measurement with the highest accuracy among the single-shot 3D measurements so far for inheriting of the traditional optional N-step PMP. And the optional larger N can result in higher accuracy while the single-shot feature can be always guaranteed both the static and real-time 3D measurements. Furthermore, the mathematical model of the proposed method is more concise compared with the traditional PMP for the phase just modulated by the measured object itself can be directly solved without two phase resolutions and two phase unwrappings for both the reference plane and the measured object. The simulations and experimental results show the feasibility and validity of the proposed method.

Ultra-high field (10.5T) diffusion-weighted MRI of the macaque brain

Diffu0sion-weighted magnetic resonance imaging (dMRI) is a non-invasive imaging technique that provides information about the barriers to the diffusion of water molecules in tissue. In the brain, this information can be used in several important ways, including to examine tissue abnormalities associated with brain disorders and to infer anatomical connectivity and the organization of white matter bundles through the use of tractography algorithms. However, dMRI also presents certain challenges. For example, historically, the biological validation of tractography models has shown only moderate correlations with anatomical connectivity as determined through invasive tract-tracing studies. Some of the factors contributing to such issues are low spatial resolution, low signal-to-noise ratios, and long scan times required for high-quality data, along with modeling challenges like complex fiber crossing patterns. Leveraging the capabilities provided by an ultra-high field scanner combined with denoising, we have acquired whole-brain, 0.58 mm isotropic resolution dMRI with a 2D-single shot echo planar imaging sequence on a 10.5 Tesla scanner in anesthetized macaques. These data produced high-quality tractograms and maps of scalar diffusion metrics in white matter. This work demonstrates the feasibility and motivation for in-vivo dMRI studies seeking to benefit from ultra-high fields.

Composite fringe projection deep learning profilometry for single-shot absolute 3D shape measurement.

Single-shot fringe projection profilometry (FPP) is essential for retrieving the absolute depth information of the objects in high-speed dynamic scenes. High-precision 3D reconstruction using only one single pattern has become the ultimate goal in FPP. The frequency-multiplexing (FM) method is a promising strategy for realizing single-shot absolute 3D measurement by compounding multi-frequency fringe information for phase unwrapping. In order to solve the problem of serious spectrum aliasing caused by multiplexing schemes that cannot be removed by traditional spectrum analysis algorithms, we apply deep learning to frequency multiplexing composite fringe projection and propose a composite fringe projection deep learning profilometry (CDLP). By combining physical model and data-driven approaches, we demonstrate that the model generated by training an improved deep convolutional neural network can directly perform high-precision and unambiguous phase retrieval on a single-shot spatial frequency multiplexing composite fringe image. Experiments on both static and dynamic scenes demonstrate that our method can retrieve robust and unambiguous phases information while avoiding spectrum aliasing and reconstruct high-quality absolute 3D surfaces of objects only by projecting a single composite fringe image.

Automated Aircraft Dent Inspection via a Modified Fourier Transform Profilometry Algorithm.

The search for dents is a consistent part of the aircraft inspection workload. The engineer is required to find, measure, and report each dent over the aircraft skin. This process is not only hazardous, but also extremely subject to human factors and environmental conditions. This study discusses the feasibility of automated dent scanning via a single-shot triangular stereo Fourier transform algorithm, designed to be compatible with the use of an unmanned aerial vehicle. The original algorithm is modified introducing two main contributions. First, the automatic estimation of the pass-band filter removes the user interaction in the phase filtering process. Secondly, the employment of a virtual reference plane reduces unwrapping errors, leading to improved accuracy independently of the chosen unwrapping algorithm. Static experiments reached a mean absolute error of at a distance of , while dynamic experiments showed at a distance of . On average, the mean absolute error decreased by , proving the validity of the proposed single-shot 3D reconstruction algorithm and suggesting its applicability for future automated dent inspections.

Three-dimensional light-field microendoscopy with a GRIN lens array.

Optical endoscopy has emerged as an indispensable clinical tool for modern minimally invasive surgery. Most systems primarily capture a 2D projection of the 3D surgical field. Currently available 3D endoscopes can restore stereoscopic vision directly by projecting laterally shifted views of the operating field to each eye through 3D glasses. These tools provide surgeons with informative 3D visualizations, but they do not enable quantitative volumetric rendering of tissue. Therefore, advanced tools are desired to quantify tissue tomography for high precision microsurgery or medical robotics. Light-field imaging suggests itself as a promising solution to the challenge. The approach can capture both the spatial and angular information of optical signals, permitting the computational synthesis of the 3D volume of an object. In this work, we present GRIN lens array microendoscopy (GLAM), a single-shot, full-color, and quantitative 3D microendoscopy system. GLAM contains integrated fiber optics for illumination and a GRIN lens array to capture the reflected light field. The system exhibits a 3D resolution of ∼100 µm over an imaging depth of ∼22 mm and field of view up to 1 cm2. GLAM maintains a small form factor consistent with the clinically desirable design, making the system readily translatable to a clinical prototype.

Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement

Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects. For fringe projection profilometry (FPP), however, it is still challenging to recover accurate 3D shapes of isolated objects by a single fringe image. In this paper, we demonstrate that the deep neural networks can be trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies. The extracted phase is free from spectrum-aliasing problem which is hard to avoid for traditional spatial-multiplexing methods. Experiments on both static and dynamic scenes show that the proposed approach is robust to object motion and can obtain high-quality 3D reconstructions of isolated objects within a single fringe image.

Single-shot 3D endoscopic imaging exploiting a diffuser and neural networks

Lens-based endoscopes offer high lateral resolution, but suffer from rigid imaging properties, such as a fixed focal plane. We present a miniaturized 0.5 mm diameter endoscope in which the objective lens is replaced by an optical diffuser. The intensity information of the object space is scattered and passed to a camera via a coherent fibre bundle. The image is reconstructed by a neural network. The field of view and resolution depend on the object distance. 3D-single-shot imaging up to video rate can be enabled. The approach shows great potential for applications like robust 3D fluorescence imaging.

Historical development of FINCH from the beginning to single-shot 3D confocal imaging beyond optical resolution [Invited

We chronicle a 15-year development effort of Fresnel incoherent correlation holography (FINCH) since its first description to its current 3D current microscopic wide-field or confocal imaging that doubles optical resolution beyond the Rayleigh limit to about 100 nm in a single snapshot. The path from the original demonstration of FINCH [Opt. Lett.32, 912 (2007) OPLEDP0146-959210.1364/OL.32.000912] to its current picture-perfect imaging of multicolor fluorescent biological specimens and reference test patterns by fluorescence or reflected light imaging is described.

Three-dimensional holographic parallel focusing with feedback control for femtosecond laser processing

We demonstrate feedback control of three-dimensional (3D) parallel focusing for automatic compensation of imperfections in an optical system and long-term stability of parallel focusing. These are realized by a computer-generated hologram (CGH) displayed on a spatial light modulator (SLM), and an iterative calculation based on the observation of the 3D focusing intensities with an image sensor on a programmable linear stage. We demonstrate our parallel laser processing method using 40 parallel beams focused at four different planes, with a maximum uniformity of 0.96, applied to single-shot 3D fabrication in transparent glass.

Advances in spiral fMRI: A high-resolution study with single-shot acquisition

Spiral fMRI has been put forward as a viable alternative to rectilinear echo-planar imaging, in particular due to its enhanced average k-space speed and thus high acquisition efficiency. This renders spirals attractive for contemporary fMRI applications that require high spatiotemporal resolution, such as laminar or columnar fMRI. However, in practice, spiral fMRI is typically hampered by its reduced robustness and ensuing blurring artifacts, which arise from imperfections in both static and dynamic magnetic fields.Recently, these limitations have been overcome by the concerted application of an expanded signal model that accounts for such field imperfections, and its inversion by iterative image reconstruction. In the challenging ultra-high field environment of 7 Tesla, where field inhomogeneity effects are aggravated, both multi-shot and single-shot 2D spiral imaging at sub-millimeter resolution was demonstrated with high depiction quality and anatomical congruency.In this work, we further these advances towards a time series application of spiral readouts, namely, single-shot spiral BOLD fMRI at 0.8 mm in-plane resolution. We demonstrate that high-resolution spiral fMRI at 7 T is not only feasible, but delivers both excellent image quality, BOLD sensitivity, and spatial specificity of the activation maps, with little artifactual blurring. Furthermore, we show the versatility of the approach with a combined in/out spiral readout at a more typical resolution (1.5 mm), where the high acquisition efficiency allows to acquire two images per shot for improved sensitivity by echo combination.

Single-shot real-time compressed ultrahigh-speed imaging enabled by a snapshot-to-video autoencoder

Single-shot 2D optical imaging of transient scenes is indispensable for numerous areas of study. Among existing techniques, compressed optical-streaking ultrahigh-speed photography (COSUP) uses a cost-efficient design to endow ultrahigh frame rates with off-the-shelf CCD and CMOS cameras. Thus far, COSUP’s application scope is limited by the long processing time and unstable image quality in existing analytical-modeling-based video reconstruction. To overcome these problems, we have developed a snapshot-to-video autoencoder (S2V-AE)—which is a deep neural network that maps a compressively recorded 2D image to a movie. The S2V-AE preserves spatiotemporal coherence in reconstructed videos and presents a flexible structure to tolerate changes in input data. Implemented in compressed ultrahigh-speed imaging, the S2V-AE enables the development of single-shot machine-learning assisted real-time (SMART) COSUP, which features a reconstruction time of 60 ms and a large sequence depth of 100 frames. SMART-COSUP is applied to wide-field multiple-particle tracking at 20,000 frames per second. As a universal computational framework, the S2V-AE is readily adaptable to other modalities in high-dimensional compressed sensing. SMART-COSUP is also expected to find wide applications in applied and fundamental sciences.

Comparative analysis of laser and simulated speckle pattern for single shot 3D reconstruction

Stereo photogrammetry is a fundamental technique for 3D measurements in different applications. As passive stereo methods struggle in texture-less regions, different light-coded patterns are projected to solve the correspondence problem. When the reconstruction method requires a series of sequentially coded light patterns, the system is prone to movement-related errors. A single pattern is utilized where the potential subjects are dynamic or change the position rapidly. A random speckle pattern is a good candidate for such systems. Primarily, there are two approaches to generate the speckle pattern in stereoscopic systems. The speckles can be generated by the coherent illumination of a laser diode, and the laser speckle-like pattern can also be simulated and projected by a professional projector. The purpose of both is to solve the stereo correspondence problem; however, the performance of both can differ, subjective to employed 3D sensing algorithm. In this Letter, we compare the performance of both for single-shot 3D reconstruction. We have discussed the pros and cons of both methods and analyze their ability to resolve long range 3D reconstruction.

Single-shot 3D tracking based on polarization multiplexed Fourier-phase camera

For moving objects, 3D mapping and tracking has found important applications in the 3D reconstruction for vision odometry or simultaneous localization and mapping. This paper presents a novel camera architecture to locate the fast-moving objects in four-dimensional (4D) space ( x , y , z , t ) through a single-shot image. Our 3D tracking system records two orthogonal fields-of-view (FoVs) with different polarization states on one polarization sensor. An optical spatial modulator is applied to build up temporal Fourier-phase coding channels, and the integration is performed in the corresponding CMOS pixels during the exposure time. With the 8 bit grayscale modulation, each coding channel can achieve 256 times temporal resolution improvement. A fast single-shot 3D tracking system with 0.78 ms temporal resolution in 200 ms exposure is experimentally demonstrated. Furthermore, it provides a new image format, Fourier-phase map, which has a compact data volume. The latent spatio-temporal information in one 2D image can be efficiently reconstructed at relatively low computation cost through the straightforward phase matching algorithm. Cooperated with scene-driven exposure as well as reasonable Fourier-phase prediction, one could acquire 4D data ( x , y , z , t ) of the moving objects, segment 3D motion based on temporal cues, and track targets in a complicated environment.

3D single-shot ptychography with highly tilted illuminations.

A method based on highly tilted illumination and non-paraxial iterative computation is proposed to improve the image quality of single-shot 3D ptychography. A thick sample is illuminated with a cluster of laser beams that are separated by large enough angles to record each diffraction pattern distinctly in a single exposure. 3D structure of the thick sample is accurately reconstructed from recorded diffraction patterns using a modified multi-slice algorithm to process non-paraxial illumination. Sufficient number of recorded diffraction patterns with noticeably low crosstalk enhances the fidelity of reconstruction significantly over single-shot 3D ptychography methods that are based on paraxial illumination. Experimental observations guided by the results of numerical simulations show the feasibility of the proposed method.

Palm-sized single-shot phase-shifting incoherent digital holography system

We propose a palm-sized single-shot phase-shifting incoherent digital holography (SSPS-IDH) system in which three-dimensional (3D) information of spatially incoherent light is recorded as a self-interference hologram. To implement this single-shot 3D recording system, we adopt thin optical elements and no refractive lens. 3D information is obtained with a single-shot exposure of a polarization image sensor. More than 3,800-fold downsizing in volume can be achieved in comparison with a conventional incoherent digital holography (DH) system. An SSPS-IDH system is constructed, and its single-shot 3D sensing ability is experimentally demonstrated.