Single Speckle Pattern Research Articles

Real-time and accurate monocular 3D sensor using the reference plane calibration and an optimized SGM based on opencl acceleration

Speckle projection profilometry (SPP), as a promising structured light projection technique, can achieve global unambiguous 3D measurement by projecting a single random speckle pattern. In addition, the projected speckle pattern is usually etched into the microstructure of highly integrated Vertical-Cavity Surface-Emitting Laser (VCSEL), which makes the hardware system compact enough to be mounted on mobile devices such as robots. However, since the stereo matching algorithm used in SPP involves high computational overhead, it usually runs in real-time on specially customized hardware platforms such as ASIC/FPGA, rather than general-purpose mobile platforms. In this paper, we propose a real-time and accurate 3D measurement method using a monocular 3D sensor based on the infrared speckle projection. Similar to Kinect v1, our sensor mainly consists of an IR dot projector and one IR camera for projecting and capturing speckle images synchronously. Low-cost and high-quality speckle projection is achieved by customizing the projection pattern of VCSEL and using the beam copy function of Diffractive Optical Elements (DOE). Based on the 3D imaging principle of monocular 3D sensors, a reference plane calibration method is proposed to obtain a high-quality reference speckle image for improving the monocular matching accuracy. Then, benefited from the local memory mechanism and multiple operating synchronizations on the OpenCL environment, an optimized semi-global matching (SGM) algorithm using GPU is presented to achieve efficient and accurate depth reconstruction dynamically. Within the measurement range of 0.8m (length) ×0.5m (width) ×1m (depth), the proposed method can achieve real-time and single-shot 3D imaging with an accuracy of 1.277 mm at 75 FPS on GTX 1060 and 15 FPS on ARM Mail G52(mobile platform).

Single-shot 3D shape measurement using an end-to-end stereo matching network for speckle projection profilometry.

Speckle projection profilometry (SPP), which establishes the global correspondences between stereo images by projecting only a single speckle pattern, has the advantage of single-shot 3D reconstruction. Nevertheless, SPP suffers from the low matching accuracy of traditional stereo matching algorithms, which fundamentally limits its 3D measurement accuracy. In this work, we propose a single-shot 3D shape measurement method using an end-to-end stereo matching network for SPP. To build a high-quality SPP dataset for training the network, by combining phase-shifting profilometry (PSP) and temporal phase unwrapping techniques, high-precision absolute phase maps can be obtained to generate accurate and dense disparity maps with high completeness as the ground truth by phase matching. For the architecture of the network, a multi-scale residual subnetwork is first leveraged to synchronously extract compact feature tensors with 1/4 resolution from speckle images for constructing the 4D cost volume. Considering that the cost filtering based on 3D convolution is computationally costly, a lightweight 3D U-net network is proposed to implement efficient 4D cost aggregation. In addition, because the disparity maps in the SPP dataset should have valid values only in the foreground, a simple and fast saliency detection network is integrated to avoid predicting the invalid pixels in the occlusions and background regions, thereby implicitly enhancing the matching accuracy for valid pixels. Experiment results demonstrated that the proposed method improves the matching accuracy by about 50% significantly compared with traditional stereo matching methods. Consequently, our method achieves fast and absolute 3D shape measurement with an accuracy of about 100µm through a single speckle pattern.

Spectral speckle-correlation imaging.

We present a method for single-shot spectrally resolved imaging through scattering media by using the spectral memory effect of speckles. In our method, a single speckle pattern from a multi-colored object is captured through scattering media with a monochrome image sensor. The color object is recovered by correlation of the captured speckle and a three-dimensional phase retrieval process. The proposed method was experimentally demonstrated by using point sources with different emission spectra located between diffusers. This study paves the way for non-invasive and low-cost spectral imaging through scattering media.

Interferometric speckle visibility spectroscopy (ISVS) for human cerebral blood flow monitoring

Infrared light scattering methods have been developed and employed to non-invasively monitor human cerebral blood flow (CBF). However, the number of reflected photons that interact with the brain is low when detecting blood flow in deep tissue. To tackle this photon-starved problem, we present and demonstrate the idea of interferometric speckle visibility spectroscopy (ISVS). In ISVS, an interferometric detection scheme is used to boost the weak signal light. The blood flow dynamics are inferred from the speckle statistics of a single frame speckle pattern. We experimentally demonstrated the improvement in the measurement of fidelity by introducing interferometric detection when the signal photon number is low. We apply the ISVS system to monitor the human CBF in situations where the light intensity is ∼100-fold less than that in common diffuse correlation spectroscopy (DCS) implementations. Due to the large number of pixels (∼2 × 105) used to capture light in the ISVS system, we are able to collect a similar number of photons within one exposure time as in normal DCS implementations. Our system operates at a sampling rate of 100 Hz. At the exposure time of 2 ms, the average signal photoelectron number is ∼0.95 count/pixel, yielding a single pixel interferometric measurement signal-to-noise ratio (SNR) of ∼0.97. The total ∼2 × 105 pixels provide an expected overall SNR of 436. We successfully demonstrate that the ISVS system is able to monitor the human brain pulsatile blood flow, as well as the blood flow change when a human subject is doing a breath-holding task.

Speckle-based determination of the polarisation state of single and multiple laser beams

Laser speckle is generated by the multiple interference of light through a disordered medium. Here we study the premise that the speckle pattern retains information about the polarisation state of the incident field. We analytically verify that a linear relation exists between the Stokes vector of the light and the resulting speckle pattern. As a result, the polarisation state of a beam can be measured from the speckle pattern using a transmission matrix approach. We perform a quantitative analysis of the accuracy of the transmission matrix method to measure randomly time-varying polarisation states. In experiment, we find that the Stokes parameters of light from a diode laser can be retrieved with an uncertainty of 0.05 using speckle images of 150×150 pixels and 17 training states. We show both analytically and in experiment that this approach may be extended to the case of more than one laser field, demonstrating the measurement of the Stokes parameters of two laser beams simultaneously from a single speckle pattern and achieving the same uncertainty of 0.05.

基于深度学习的散斑投影轮廓术

Traditional single speckle pattern matching algorithms always suffer from the low measurement accuracy and cannot be used to measure complex surface objects. A speckle projection profilometry with deep learning was proposed to realize the pixel-by-pixel matching. The siamese convolutional neural network structure was applied and extended where the main speckle pattern and the auxiliary speckle pattern were fed into the neural network patch by patch. It was expected that the feature from the speckle pattern patches could be extracted by the convolution operation. In this way, the features were fused and the matching coefficient between the two patches was obtained, which could be further used to formulate the disparity data and then the three-dimensional (3D) object was reconstructed. The experiment results demonstrate that with the proposed method 3D measurement with an accuracy of about 290 μm could be achieved through a single speckle pattern.

Photonic human identification based on deep learning of back scattered laser speckle patterns.

The analysis of the dynamics of speckle patterns that are generated when laser light is back scattered from a tissue has been recently shown as very applicable for remote sensing of various bio-medical parameters. In this work, we present how the analysis of a static single speckle pattern scattered from the forehead of a subject, together with advanced machine learning techniques based on multilayered neural networks, can offer novel approach to accurate identification within a small predefined number of classes (e.g., a 'smart home' setting which restricts its operations for family members only). Processing the static scattering speckle pattern by neural networks enables extraction of unique features with no previous expert knowledge being required. Using the right model allows for a very accurate differentiation between desirable categories, and that model can form a basis for using speckles patterns as a form of identity measure of 'forehead-print'.

An effective compressive computational ghost imaging with hybrid speckle pattern

As an important factor to determine the quality of imaging, the sizes of the speckle pixel have gained increasing attention in ghost imaging. This paper proposes a new compressive computational ghost imaging which using hybrid speckle pattern to retrieve the target area. The hybrid speckle pattern is composed of different sizes of speckle pixel which depends on the resolution of target area. This method consists of two parts, the first part is the determination of interesting region and the second part is the imaging. The ghost images with hybrid speckle pattern are reconstructed by using CCGI. Simulation and experiment results have demonstrated the effectiveness and feasibility of the proposed method. The results indicate that the structural similarity index measure and signal-to-noise ratio are increased remarkably compared with single speckle pattern.

Different methods for characterizing surface roughness using laser speckle technique

In this work, results from an optical technique (laser speckle technique) for measuring surface roughness was done by using statistical properties of speckle pattern from the point of view of computer image texture analysis. Four calibration relationships were used to cover wide range of measurement with the same laser speckle technique. The first one is based on intensity contrast of the speckle, the second is based on analysis of speckle binary image, the third is on size of speckle pattern spot, and the latest one is based on characterization of the energy feature of the gray level co-occurrence matrices for the speckle pattern. By these calibration relationships surface roughness of an object surface can be evaluated within these relations ranges from single speckle pattern image which was taken from the surface.

Single-shot diffraction-limited imaging through scattering layers via bispectrum analysis.

Recently introduced speckle correlations-based techniques enable noninvasive imaging of objects hidden behind scattering layers. In these techniques, the hidden object Fourier amplitude is retrieved from the scattered light autocorrelation, and the lost Fourier phase is recovered via iterative phase-retrieval algorithms, which suffer from convergence to wrong local minimums solutions and cannot solve ambiguities in object orientation. Here, inspired by notions used in astronomy, we experimentally demonstrate that in addition to Fourier amplitude, the object-phase information is naturally and inherently encoded in the scattered light bispectrum (the Fourier transform of triple correlation) and can also be extracted from a single high-resolution speckle pattern, based on which we present a single-shot imaging scheme to deterministically and unambiguously retrieve diffraction-limited images of objects hidden behind scattering layers.

Novel 3D measurement system based on speckle and fringe pattern projection.

An efficient three-dimensional shape measurement system is proposed, which is based on the combining projection of single digital speckle pattern and phase-shifting fringe patterns. At the beginning, the initial corresponding point for each pixel is obtained by a novel speckle-phase combination method. The initial information can be calculated by the single speckle pattern in a short time, while the phase information is used to ensure the results. Unlike the conventional methods, it is not necessary to obtain the unwrapped phase, therefore the number of projected patterns is reduced greatly. Then accurate corresponding coordinates are obtained according to the wrapped phase. Three cases are analyzed while adjusting the initial corresponding coordinates locally. Thus accuracy coordinates are obtained without missing or incorrect points. Experiments demonstrate that we can achieve accurate reconstruction results with reduced measurement time by the proposed method.

Skeleton extraction and phase interpolation for single ESPI fringe pattern based on the partial differential equations.

A novel phase extraction method for single electronic speckle pattern interferometry (ESPI) fringes is proposed. The partial differential equations (PDEs) are used to extract the skeletons of the gray-scale fringe and to interpolate the whole-field phase values based on skeleton map. Firstly, the gradient vector field (GVF) of the initial fringe is adjusted by an anisotropic PDE. Secondly, the skeletons of the fringe are extracted combining the divergence property of the adjusted GVF. After assigning skeleton orders, the whole-field phase information is interpolated by the heat conduction equation. The validity of the proposed method is verified by computer-simulated and experimentally obtained poor-quality ESPI fringe patterns.

Frame-rate analysis of arterial blood flow in human and rat using laser speckle image sensing

In imaging of blood flow by means of a laser speckle technique, we have proposed so far an estimation parameter based on the spatial contrast of speckle patterns observed for the blood flow in skin tissue and a blood vessel. This parameter enable us to image a relative blood flow distribution from a single speckle pattern, thus, it analyzes the blood flow with a frame-rate of an imaging device used. In this study, we investigated availability of this parameter for detecting changes in arterial blood flow caused by medication and cold stimulation to the skin tissue. Experiments were conducted for an anesthetized rat and a human wrist to confirm the feasibility of the present parameter.

Application of the radial basis function interpolation to phase extraction from a single electronic speckle pattern interferometric fringe

In this paper, we introduce the radial basis function (RBF) interpolation method to electronic speckle pattern interferometry (ESPI) and propose the RBF interpolation method to obtain unwrapped phase values based on a skeleton map. Because of the excellent approximation properties of the RBF interpolation, the proposed method can extract accurate phase values from a single fringe pattern effectively, even using a simple 3×3 mean filter as preprocessing. Using our method, both special filtering methods for ESPI fringes as preprocessing and postprocessing, including a dilatation and erosion algorithm for pruning and connecting and the smooth algorithm for improving the phase values are not needed. We test our method on a computer-simulated and two experimentally obtained poor-quality fringe patterns. The results have demonstrated that our RBF interpolation method works well even under a seriously disconnected skeleton map where it is impossible to apply the widely used, Matlab function grid data interpolation or the backpropagation neural networks method [Appl. Opt. 46, 7475 (2007)].

Grinding surface roughness measurement based on the co-occurrence matrix of speckle pattern texture

Surface speckle pattern intensity distribution resulting from laser light scattering from a rough surface contains various information about the surface geometrical and physical properties. A surface roughness measurement technique based on the texture analysis of surface speckle pattern texture images is put forward. In the surface roughness measurement technique, the speckle pattern texture images are taken by a simple setup configuration consisting of a laser and a CCD camera. Our experimental results show that the surface roughness contained in the surface speckle pattern texture images has a good monotonic relationship with their energy feature of the gray-level co-occurrence matrices. After the measurement system is calibrated by a standard surface roughness specimen, the surface roughness of the object surface composed of the same material and machined by the same method as the standard specimen surface can be evaluated from a single speckle pattern texture image. The robustness of the characterization of speckle pattern texture for surface roughness is also discussed. Thus the surface roughness measurement technique can be used for an in-process surface measurement.

Phase-shifting speckle interferometry

Speckle patterns have high frequency phase data, which make it difficult to find the absolute phase of a single speckle pattern; however, the phase of the difference between two correlated speckle patterns can be determined. This is done by applying phase-shifting techniques to speckle interferometry, which will quantitatively determine the phase of double-exposure speckle measurements. The technique uses computer control to take data and calculate phase without an intermediate recording step. The randomness of the speckle causes noisy data points which are removed by data processing routines. One application of this technique is finding the phase of deformations, where up to ten waves of wavefront deformation can easily be measured. Results of deformations caused by tilt of a metal plate and a disbond in a honeycomb structure brazed to an aluminum plate are shown.