Fourier Transform Profilometry Research Articles

Real-time 3D shape measurement of dynamic scenes using fringe projection profilometry: lightweight NAS-optimized dual frequency deep learning approach.

Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.

Estimation of tension distribution on drumheads by visualization of high-order modes using Fourier transform profilometry

In this study, higher vibration modes of drumheads were visualized using Fourier transform profilometry (FTP) for simplifying drum tuning process. In our previous work, we confirmed that images of the (1,1) mode of a drumhead had some curved nodal lines due to non-uniform tension and that it provided visual cues on the state of drum tuning. However, when there were two areas with low tension on a drumhead, non-uniform tension did not curve the nodal lines. Therefore, focusing only on the (1,1) mode can lead to misinterpreting a non-uniform state as uniform. In this research, we expand the scope of visualization to higher modes in order to alleviate this problem. If nodal lines can be visualized even in higher modes, information not included in the (1,1) mode can be obtained, which will allow us to know the tuning state more faithfully. In a visualization experiment, we focused mainly on the (2,1) mode, which consists of two nodal lines. FTP-based visualization confirmed that even when the tension distribution had two areas with low tension, those nodal lines curved around each area. Based on this, it became able to estimate tension distribution that had been challenging with only the (1,1) mode considered.

Performance of Background Oriented Schlieren with different background patterns and image processing techniques

Synthetic tests of the accuracy, spatial resolution and robustness with respect to large gradient of the apparent displacement are performed for Background Oriented Schlieren (BOS) measurements using various background patterns and image processing techniques, including multi-pass cross-correlation, iterative Horn–Schunck and Lucas–Kanade optical flow algorithms and Fourier Transform Profilometry (FTP). It is shown that FTP with periodic sinusoidal or binary fringe pattern outperforms all the other methods. With optimal pattern period λpattern=6−15pix it is able to resolve perturbations with wavelength down to λpattern2 and to obtain reliable results for images with displacement gradient up to 0.9 pix/pix. Better performance of FTP, compared to multi-pass cross-correlation, is confirmed for real experimental measurements of water temperature in natural convection near a heated vertical plate. FTP advantages and limitations, including possible nonlocal error, associated with phase unwrapping, are discussed.

Vision-Based Structural Modal Identification Using Hybrid Motion Magnification

As a promising alternative to conventional contact sensors, vision-based technologies for a structural dynamic response measurement and health monitoring have attracted much attention from the research community. Among these technologies, Eulerian video magnification has a unique capability of analyzing modal responses and visualizing modal shapes. To reduce the noise interference and improve the quality and stability of the modal shape visualization, this study proposes a hybrid motion magnification framework that combines linear and phase-based motion processing. Based on the assumption that temporal variations can represent spatial motions, the linear motion processing extracts and manipulates the temporal intensity variations related to modal responses through matrix decomposition and underdetermined blind source separation (BSS) techniques. Meanwhile, the theory of Fourier transform profilometry (FTP) is utilized to reduce spatial high-frequency noise. As all spatial motions in a video are linearly controllable, the subsequent phase-based motion processing highlights the motions and visualizes the modal shapes with a higher quality. The proposed method is validated by two laboratory experiments and a field test on a large-scale truss bridge. The quantitative evaluation results with high-speed cameras demonstrate that the hybrid method performs better than the single-step phase-based motion magnification method in visualizing sound-induced subtle motions. In the field test, the vibration characteristics of the truss bridge when a train is driving across the bridge are studied with a commercial camera over 400 m away from the bridge. Moreover, four full-field modal shapes of the bridge are successfully observed.

Fourier transform profilometry using single-pixel detection based on two-dimensional discrete cosine transform

We present a single-pixel Fourier transform profilometry (FTP) for obtaining three-dimensional (3D) maps with few measurements. Two digital micromirror devices were employed to generate a two-dimensional (2D) sinusoidal illumination pattern and a series of specified sampling patterns, respectively, coupled with a single-pixel detector to sequentially collect spatially light field from an object surface. The reflectance image at the object surface was recovered with single-pixel imaging (SPI) scheme, 3D map of the object surface was reconstructed with FTP from the recovered reflectance image. We incorporated the 2D discrete cosine transform into the SPI scheme to reduce the number of sampling patterns, where only a few sampling patterns whose spatial frequencies around the illumination frequency were required for the measurement. We experimentally reconstructed acceptable 3D maps, the ratios of the number of measurements to the number of pixels of 3D maps were determined to be 1.04% and 1.79% for the 3D reconstructions of simple and complex objects, respectively. This significantly reduces the number of measurements that is necessary for the current 3D reconstruction based on SPI scheme.

Cross-shaped Hanning filter used in Fourier transform profilometry for accurate 3-D shape retrieval

Fourier transform profilometry (FTP) is widely used for real-time three-dimensional (3-D) surface shape measurement with a single frame of projection fringe pattern. The band-pass filter is very important to the shape retrieval accuracy of this technique. On the basis of the Hanning band-pass filter, a cross-shaped Hanning filter is developed in the paper to extract the first-order spectrum of deformed fringe pattern, and the effect of fringe angle on the proposed filter is analysed. The phase retrieval results using the proposed filter and Hanning filter are compared in the simulation and measurement experiment. The results show that the cross-shaped Hanning filter coordinating with the fringe pattern of about 45 degrees holds the higher accuracy of 3-D shape retrieval.

PMENet: phase map enhancement for Fourier transform profilometry using deep learning

Fringe projection profilometry (FPP) is a three-dimensional (3D) shape measurement method that involves projecting fringe patterns onto the object. A phase map retrieved from these fringe images is used for reconstructing the 3D surface of the object. Fourier transform and phase-shifting are two of the widely used fringe analysis techniques for performing 3D shape measurement using FPP. Fourier transform profilometry (FTP) has the advantage of performing high-speed measurement due to its single-shot nature. However, the reconstructed 3D surface has artifacts and high noise, specifically on the edges. On the other hand, phase-shifting profilometry (PSP) has the advantage of higher accuracy (relatively less noise level) but compromises on measurement speed. In this research, we propose a deep learning method to enhance the quality of the FTP phase maps using an efficient deep learning model called phase map enhancement net (PMENet). PMENet takes an FTP phase map as input and predicts a high-quality phase map in a supervised manner by using the phase maps obtained from 18-step PSP as ground truth. The training dataset was generated using a virtual FPP system. Validations were conducted on both the simulated data (generated by virtual FPP system) and the real-world data. The experimental results demonstrate that the trained neural network model can successfully improve the quality of the 3D geometric reconstruction with FTP and reduced the mean and root-mean-square error by 66% and 43%, respectively.

Embedded optical fiber link interferometer sensors for snapshot surface inspection using the synthetic wavelength technique.

Free-space optical measurement systems can have a direct impact on evaluation systems operational in propagation paths. During propagation via optical fibers, light suffers scattering or interference, causing some output signal loss with an uncertainty outcome. Therefore, this study aims to explore the instant decisions related to the use of single- and multi-mode fiber optics and how they affect the gathering of data from high-speed optical measurement instrument links. The study also seeks to address a number of design methodology aspects and the empirical outcomes related to a surface topography measurement sensor based on fiber optics capable of surface roughness or step-height measurement. The study suggests that the Fourier transform profilometry method (FTP) can overcome the disadvantages of optical metrology sensors (e.g., bulkiness, challenging set-up, high costs, and low speed). However, despite eliminating vertical height problems, the Fourier transform profilometry (FTP) did have some shortcomings for every outcome related to core variables, including the dispersive optical fiber link sensor. The synthetic wavelength method enabled the dispersive optical fiber link sensor to calculate the vertical step height of the selected sample (1 µm). There was improved step-height repeatability, with satisfactory from 20 to 18 nm outcome improvement range. Additional investigations are necessary to establish the compatibility of single- or multi-mode optical fiber sensors with particular instruments, especially those currently preferred for embedded metrology applications.

High dynamic range real-time 3D measurement based on Fourier transform profilometry

The high dynamic range (HDR) fringe projection technique is widely used to measure the three-dimensional (3D) shapes of objects with shiny surfaces; however, it tends to compromise when measuring the dynamic objects. To reduce the number of images and improve the real-time performance, a Fourier transform profilometry (FTP)-based method that can increase the dynamic range for real-time 3D measurement is proposed. Firstly, single-color patterns are projected onto the measured object, and one captured color image can be separated into three monochrome images with different intensities by taking advantage of the different responses of R, G, and B channels. Second, the HDR fringe pattern is generated by compositing three monochrome images, and then processed by the background-normalized algorithm to obtain the final fringe pattern. Subsequently, FTP is employed to retrieve the phase map and obtain the 3D shape. The experimental results demonstrated that the proposed method can measure the dynamic objects with shiny surfaces in real-time 3D measurement system.

Low-Frequency Vibration Measurement Based on the Concentric-Circle Grating Projection System

In low-frequency vibration measurement field, optical dynamic 3-D measurement based on camera-projector system is a novel approach. In this system, linear sinusoidal grating is the most common projection mode, which can meet the requirements of some vibration measurement. However, the dynamic measurement range of amplitude using the combination of linear grating and Fourier transform profilometry (FTP) is limited in some vibration measurement environments, because the linear fringe is periodic. To solve the problem, this article proposes a low-frequency vibration measurement method based on the concentric-circle grating camera-projector system and FTP. The center of the circle is traced dynamically and the coordinate system is transformed with the center of the circle as the origin. The phase distribution of concentric-circle grating is obtained by coordinate system transformation and FTP. By exploring the relationship between phase and relative height, the time-varying curve of the object surface height is recovered, which contains the vibration information. The experiments compare the measurement ability of the concentric-circle and linear grating. The results show that the linear grating and concentric-circle grating with the similar amplitude resolution have different amplitude measurement range. In this experiment, the maximum measurement range of linear grating is about 2.4 mm, while that of concentric-circle grating is more than about 3.9 mm. It means that the concentric-circle grating projection method can measure a large dynamic range of amplitude under the condition that the center of the circle can be traced, and solve the problem of amplitude limitation of linear grating in the use of FTP.

Improving the Performance of 3D Shape Measurement of Moving Objects by Fringe Projection and Data Fusion

Fringe projection profilometry (FPP) has been widely studied for measurements of three-dimensional (3D) shapes. However, the measurement of moving objects is still a challenging problem associated with commonly used methods. Phase-shifting profilometry (PSP) suffers from motion-induced errors when applied to moving objects, while Fourier transform profilometry (FTP) is sensitive to background light and noise. This paper proposes a novel data fusion approach to improve the measurement of moving objects by alleviating noise-induced errors. Firstly, a set of coarse measurements on the shape of the object are obtained at different time instants by FTP. Then the parameters associated with the motion of the object over these time instants are retrieved from these coarse estimates by applying the iterative closest point (ICP) method. Finally, the parameters are utilized to adaptively fuse the coarse measurements based on their quality. Simulations and experiments verify that the proposed approach can significantly improve the measurement accuracy.

Two-step phase shifting profilometry based on Lissajous ellipse fitting technique

<sec> Phase shifting profilometry (PSP) is an effective technique to reconstruct the three-dimensional shape of object. In general, PSP needs three or more fringe patterns with phase-shifting accurately known to extract the phase distribution of objects. Therefore, the scene and the test objects should remain stationary during capturing the fringe patterns. However, the phase shifts may be unknown in an actual PSP measurement system, especially when measuring the moving object, that is, the phase-shift error may be introduced during the obtaining of the phase-shifting fringe patterns of moving object. In the dynamic measurement scenario, the use of fewer fringe patterns can realize the faster measurement speed and suppress the phase shift error introduced by the moving object. In this paper, a two-step PSP algorithm is proposed based on Lissajous ellipse fitting (LEF). The proposed method uses only two fringe patterns to extract the phase distribution of the object and can suppress the phase shift error caused by the moving object. </sec><sec> However, in a practical PSP system, the spatiotemporally varying background intensity and modulation also significantly affect the phase accuracy extracted by LEF, and thus three error-suppressing methods are proposed to reduce the phase error caused by the non-uniform background intensity and modulation. In order to verify the effectiveness of the three error-suppressing methods, we analyze and compare their performances of error suppression under different background intensities and modulations. The advantages of three error-suppressing methods can be summarized as follows. 1) The mean and modulation correction technique has greater advantage than the other two when the background intensity and modulation vary with time. 2) When the background intensity and modulation are relevant to pixel position and the number of fringe patterns, the empirical mode decomposition normalization can more effectively suppress the influence of the non-uniform background intensity and modulation. </sec><sec> In experiment, a two-step phase-shifting dynamic measurement based on LEF is conducted. Compared with the traditional PSP which needs at least three fringe patterns, the two-step PSP algorithm successfully extracts the phase with only two fringe patterns and suppresses the phase shift error caused by the motion of the object. Compared with Fourier transform profilometry (FTP), the two-step PSP algorithm can obtain very accurate phase distribution and retain many phase details. </sec>

Composite deep learning framework for absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation

Fourier transform profilometry (FTP) is a classic three-dimensional (3D) shape measurement technique that can retrieve the wrapped phase from a single fringe pattern. However, suffering from the spectral leakage and overlapping problems, it generally yields a coarse phase map with low spatial resolution and precision. Recently, deep learning has been introduced to the field of Fringe projection profilometry (FPP), revealing promising results in fringe analysis, phase unwrapping, depth constraint and system calibration. However, for absolute shape measurement of general objects, the inherent depth ambiguity problem of a single fringe is still insurmountable. In this work, we propose a composite deep learning framework for absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation. Our method combines the advantages of FPP techniques for high-resolution phase retrieval and speckle correlation approaches for robust unambiguous depth measurement. The proposed deep learning framework comprises two paths: one is a U-net-structured network, which is used to extract the wrapped phase maps from a single fringe pattern with high accuracy (but with depth ambiguities). The other stereo matching network produces the initial absolute (but with low resolution) disparity map from an additional speckle pattern. The initial disparity map is refined by exploiting the wrapped phase maps as an additional constraint and finally, a high-accuracy high-resolution disparity map for absolute 3D measurement can be obtained. Experimental results demonstrated that the proposed deep-learning-based method could realize high-precision absolute 3D measurement with an accuracy of 50 µm for measuring objects with complex surfaces.

A real-time 3D shape measurement with color texture using a monochromatic camera

High-speed real-time three-dimensional (3D) shape measurement with color texture has a large potential in various fields. The majority of the existing methods retrieve the 3D shape with color texture by employing an RGB camera or additional infrared camera; however, the RGB camera is more expensive and has lower sensitivity to light, and employing additional infrared camera requires complex alignment. Given this, a novel method is proposed to simultaneously retrieve the 3D shape and color texture. Specifically, compared with the existing methods, the proposed method only uses one monochromatic camera; as a result, the cameras alignment is not required, simplifying the 3D measurement system and reducing its cost. Three fringe patterns with different frequencies and three corresponding uniform gray-level patterns are set to different colors of red, green, and blue. Then Fourier transform profilometry (FTP) is employed to retrieve the 3D shape from the captured six images. While three uniform gray-level patterns can be used to eliminate the effects of the zero frequency term, they can be also used to generate the color texture of objects meanwhile. The proposed method only requires the capture of six monochromes, and the experimental results demonstrate the success of the proposed method.

Improve Temporal Fourier Transform Profilometry for Complex Dynamic Three-Dimensional Shape Measurement.

The high-speed three-dimensional (3-D) shape measurement technique has become more and more popular recently, because of the strong demand for dynamic scene measurement. The single-shot nature of Fourier Transform Profilometry (FTP) makes it highly suitable for the 3-D shape measurement of dynamic scenes. However, due to the band-pass filter, FTP method has limitations for measuring objects with sharp edges, abrupt change or non-uniform reflectivity. In this paper, an improved Temporal Fourier Transform Profilometry (TFTP) algorithm combined with the 3-D phase unwrapping algorithm based on a reference plane is presented, and the measurement of one deformed fringe pattern producing a new 3-D shape of an isolated abrupt objects has been achieved. Improved TFTP method avoids band-pass filter in spatial domain and unwraps 3-D phase distribution along the temporal axis based on the reference plane. The high-frequency information of the measured object can be well preserved, and each pixel is processed separately. Experiments verify that our method can be well applied to a dynamic 3-D shape measurement with isolated, sharp edges or abrupt change. A high-speed and low-cost structured light pattern sequence projection has also been presented, it is capable of projection frequencies in the kHz level. Using the proposed 3-D shape measurement algorithm with the self-made mechanical projector, we demonstrated dynamic 3-D reconstruction with a rate of 297 Hz, which is mainly limited by the speed of the camera.

Handheld pocket-size Fourier transform profilometry using projector-enabled smartphone

ABSTRACTSurface Fourier transform profilometry (FTP) is a technique used for reconstructing three-dimensional (3-D) surface topography using light. It has been widely used in machine vision for biomedical and biometric automations, providing solutions beyond conventional 2-D imaging. This paper proposes an implementation of the handheld pocket-size FTP for 3-D surface profile imaging using a projector-enabled Samsung Galaxy Beam smartphone. In the implementation, a crossed-optical-axes geometry of the FTP is formed by using a mirror positioned over the phone’s projector via an adjustable tilt mounting bracket. Experimental proof-of-concept of the proposed profilometry is done by implementing conventional and non-phase shifting FTPs with different diffuse test objects. The experimental results obtained by using the non-phase shifting technique are in good agreement with those of the direct contact measurement. Besides having superiority of compactness, the proposed profilometry paves the way for the development of real-time 3-D profiling and printing through internet or Bluetooth interconnection.

Real-Time Dynamic 3D Shape Reconstruction with SWIR InGaAs Camera.

In this paper, a real-time, dynamic three-dimensional (3D) shape reconstruction scheme based on the Fourier-transform profilometry (FTP) method is achieved with a short-wave infrared (SWIR) indium gallium arsenide (InGaAs) camera for monitoring applications in low illumination environments. A SWIR 3D shape reconstruction system is built for generating and acquiring the SWIR two-dimensional (2D) fringe pattern of the target. The depth information of the target is reconstructed by employing an improved FTP method, which has the advantages of high reconstruction accuracy and speed. The maximum error in depth for static 3D shape reconstruction is 1.15 mm for a plastic model with a maximum depth of 36 mm. Meanwhile, a real-time 3D shape reconstruction with a frame rate of 25 Hz can be realized by this system, which has great application prospects in real-time dynamic 3D shape reconstruction, such as low illumination monitoring. In addition, for real-time dynamic 3D shape reconstruction, without considering the edge areas, the maximum error in depth among all frames is 1.42 mm for a hemisphere with a depth of 35 mm, and the maximum error of the average of all frames in depth is 0.52 mm.

Synthetic wavelength to increase the snapshot optical sensor's elevated vertical measurement ranges.

Screening manufactured products that are conducted faster to enhance the contemporary manufacture processes and quality is possible by implementing enhanced quality control. Such quality control of manufactured products has increased the market for process-focused precision metrology that can execute evaluations faster while providing significant feedback for the manufacturing system. This investigation examines spatial dispersive interferometry's potential for producing accurate surface profile measurements by emphasizing vertical range measurements and identifying a system that can enable them to increase incrementally while maintaining the results' quality. Thus, this investigation selected Fourier transform profilometry (FTP) to assess surface profile measurements, as it provides the most reliable and fastest outcome data regarding this sensor. Exploring new surface scanning methods is important, as crucial weaknesses hinder several common approaches. As optical metrology sensors are bulky, difficult to establish, and expensive, the investigation will prove that FTP can resolve these restrictions. The investigation uses the synthetic wavelength approach for addressing vertical measurement limitation concerning optical systems for extending surface step height's vertical measurement range. Though it was observed that the FTP technique surmounts the vertical height limitations, certain limitations were also noted, with all outcomes considering key variables, including the scanning objective lens, system resolution, the spectrometer resolution, and diffraction grating. Future examinations must examine a wider vertical range to expand the snapshot spatial dispersive interferometry process's scope. Further, the step-height repeatability is enhanced, showing a good outcome range from 22 to 20 nm.

High Speed 3D Shape Measurement with Temporal Fourier Transform Profilometry

A novel high-speed 3D shape measurement technology called temporal Fourier transform profilometry (TFTP for short) is proposed by combining the merits of Fourier transform profilometry (FTP) and phase-measuring profilometry (PMP). Instead of using the digital light projector, a mechanical projector is employed to generate multi-period phase-shifting fringe patterns sequentially. During the reconstruction process, the phase value of each pixel is calculated independently along the temporal axis and no spectrum filtering operation is performed in a spatial domain. Therefore, high-frequency components containing the detailed information of the measured object effectively remain. The proposed method is suitable for measuring isolated dynamic objects. Only one frame of deformed fringe pattern is required to retrieve one 3D shape of the measured object, so it has the obvious advantage if measuring the dynamic scene at a high speed. A low-cost self-made mechanical projector with fast projection speed is developed to execute the principle-proof experiments, whose results demonstrate the feasibility of measuring isolated dynamic objects.

Characterization of Corneoscleral Geometry Using Fourier Transform Profilometry in the Healthy Eye.

To characterize peculiarities of the corneoscleral geometry in healthy eyes. This is a prospective case series including 88 healthy eyes of 88 patients with an age ranging from 21 to 73 years. A complete ocular examination was performed with emphasis on the analysis of the corneoscleral topographic profile with the Fourier transform profilometer Eye Surface Profiler (Eaglet-Eye BV, Houten, the Netherlands). The distribution of different topographic parameters was evaluated, as well as the correlations between corneal and scleroconjunctival parameters. Mean values of 8.64±0.37 (range, 7.81-9.50 mm), 6.06±0.52 (4.88-7.63 mm) and 11.93±1.32 mm (8.17-15.89 mm) were obtained for inner, limbal, and outer best-fit sphere, respectively. Mean values of 8.54±0.38 (7.86-9.66 mm) and 13.35±1.29 mm (11.05-17.31 mm) were obtained for mean corneal and scleral radius, respectively. Regarding tangent angles at limbus, mean values of 35.31±6.55°, 38.76±5.90°, 32.75±7.04°, and 25.91±8.99° were obtained for nasal, temporal, superior, and inferior angles, respectively. Mean difference between temporal and nasal sagittal heights increased from -1.48±120.70 μm for a chord of 11 mm to 73.53±236.55 μm for a chord of 14 mm. A weak but statistically significant correlation was found between corneal and scleral radii (r=0.325, P=0.004). The maximum sagittal height for a diameter of 12 mm was significantly correlated with flattest keratometry, astigmatism, corneal diameter, and corneal eccentricity (R: 0.77, P<0.001). The scleroconjunctival surface in the healthy eye presents some level of nasotemporal asymmetry that is higher with increasing diameters of analysis.