Color Fringe Research Articles

End-to-end color fringe depth estimation based on a three-branch U-net network

In fringe projection profilometry (FPP), end-to-end depth estimation from fringe patterns for FPP attracts more and more attention from fringe patterns. However, color images provide additional information from the RGB channel for FPP, which has been paid little attention in depth estimation. To this end, in this paper we present for the first time, to the best of our knowledge, an end-to-end network for depth estimation using color composite fringes with better performance. In order to take advantage of the color fringe pattern, a multi-branch structure is designed in this paper, which learns the multi-channel details of the object under test by using three encoders for each RGB channel and introduces an attention module to better capture the complex features and modalities information in the input data. Experiments from simulated and real datasets show that the proposed method with color fringe pattern is effective for depth estimation, and it outperforms other deep learning methods such as UNet, R2Unet, PCTNet, and DNCNN.

New high sensitivity approach for a quantitative visualization of the continuous whole-field stress evolution in digital photoelasticity

Digital photoelasticity is a widely used technique for experimental stress analysis on mechanical models subjected to loading conditions, due to its ability to reveal the stress information through color fringe patterns and consequently demodulate it computationally to achieve a quantitative evaluation. In this process, some limitations persist, such as, expertise dependence, experimental repeatability, sensitivity to load increments, and weak performance under experimental conditions, especially in time-dependent scenarios. In response to these weaknesses, the method “Quantitative Visualization of the Continuous Whole-Field Stress Evolution” introduced the ability to perform a complete demodulation of the fringe patterns generated in time-dependent applications, achieving high performance compared to conventional photoelasticity methods. This paper proposes a new highly sensitive approach for a quantitative visualization of the continuous whole-field stress evolution in digital photoelasticity by incorporating an inverse signal process and a multi-stage strategy. The algorithm aims to ensure the performance under high fringe order conditions while exhibiting high sensitivity in cases where the maximum stress obtained is less than one fringe order. The results presented validate its effectiveness in demodulating full-field stress distributions, especially in scenarios where the loading condition induces fringe orders less than one, thus contributing to the broader applicability of digital photoelasticity.

A fast three-dimensional measurement method based on color fringe projection and background intensity calibration

A fast three-dimensional measurement method based on color fringe projection and background intensity calibration

A robust phase unwrapping method for phase shifting coding high-frequency color fringe projection profilometry

Phase unwrapping is one of the key problems in fringe projection profilometry (FPP). How to achieve unambiguous fringe step order to perform phase unwrapping with few patterns still face a major challenge. We propose a robust phase unwrapping method by using a pair of encoded color fringe patterns, one is a conventional three-step phase-shifting color fringe pattern with equal phase shifts, another is added an additional phase shift valve t(k) to the former, in which t(k) is related to the order of fringe periods. A single-valued solution function for t(k) is constructed and used to determine the unique fringe order. Besides, multiplexing this pair of fringes can help denoise the extracted wrapped phase to improve the accuracy. The simulation and experimental results proved that the proposed method can achieve unambiguous phase unwrapping for high-frequency fringe patterns. The proposed method is also applicable for isolated objects.

Three-Dimensional Shape Measurement of Moving Object with Color Fringe Projection

Three-Dimensional Shape Measurement of Moving Object with Color Fringe Projection

Deep learning-based binocular composite color fringe projection profilometry for fast 3D measurements

At present, deep learning plays a crucial role in structured light 3D reconstruction. Further, in the field of fringe projection profilometry, learning 3D features from fringes and performing 3D reconstruction are being studied by many researchers. This paper combines deep learning with binocular fringe projection, uses three channels to form a single-composite-color fringe pattern for three different frequencies as the input, and predicts the numerator and denominator required to solve the wrapped phase of the object. The wrapped phase is calculated using an arctangent function. The absolute phase is obtained by unwrapping the multi-frequency heterodyne method, and the absolute phase of the left and right cameras is matched to obtain a disparity map. The parameters obtained through camera calibration can restore the 3D shape of the object, which greatly reduces the number of fringes required; accuracy close to the phase of the training set is achieved. Finally, the experimental results demonstrate the feasibility of this approach.

Joint polarization-color coding strategy based on Stokes parameter for robust strip-edge detection in 3D measurement

Stripe-edge detection of color fringe is crucial in structured light measurement. However, the robustness of color stripe-edge detection is affected by the color crosstalk, scattering, and the color filter in the color camera. This can lead to a decrease in imaging quality. In order to avoid the influence of color crosstalk, this paper analyzes the polarization properties of RGB channels and proposes a joint polarization-color coding strategy based on the Stokes parameter and a corresponding stripe-edge detection method. Compared with the traditional color-coded fringe measurement system, the monochrome camera can distinguish the color-coded fringe so that the influence induced by color crosstalk is eliminated completely. Furthermore, the influence induced by the scattering is reduced. The experiment shows that the robustness of strip-edge detection is remarkably improved, the nonparallelism of detected edges reduced by 32.4%, and the nonuniformity of detected edges reduced by 44.1%.

Surface structured light sensor with simultaneous color mapping

To obtain geometric information and color texture simultaneously, a surface structured light sensor consisting of a monochrome camera, a color camera, and a projector is proposed. The sensor uses a color camera to acquire surface color information while using it as a geometric measurement unit and matching with the monochrome camera to obtain geometric information. Due to the Bayer array and demosaicing algorithm of the color camera, pixel RGB components are always coupled with interference from other channels. However, existing color de-crosstalk in reconstruction is merely applied to the decoupling of color composite patterns, ignoring the intensity errors present in color fringe patterns under monochrome illumination. In our sensor, de-crosstalk of monochromatic patterns is considered to guarantee the reconstruction accuracy. The high-accuracy measurement of the sensor is validated by reconstructing standard steps, yielding a mean absolute error of 0.008 mm for distance measurements. In addition, the reconstruction experiment of a terracotta warrior verifies that the proposed sensor has potential application in the digital preservation of cultural relics.

Accurate calibration for crosstalk coefficient based on orthogonal color phase-shifting pattern.

The crosstalk coefficient calibration of de-crosstalk in color fringe projection profilometry is an essential step for the high-accuracy measurement. In this paper, a novel approach for calibrating crosstalk matrix of color camera is proposed. The wrapped phase error model introduced by color crosstalk in orthogonal pattern is established. Compared with the existing calibration methods depending on calculating the modulation of the crosstalk channel, the crosstalk coefficients are obtained from phase error in our method. By projecting the designed color orthogonal phase-shifting fringe patterns onto a white plate, the phase-shifting fringe patterns in both horizontal and vertical directions can be separated from captured images. The coefficients between different channels are calibrated by fitting the error relationship between the wrapped phase containing crosstalk and the standard ones. Coefficient fitting simulations and experimental validations including shape measurement of a white plate and distance measurement of a step block are carried out to verify the effectiveness of the proposed method.

Research on highly dynamic 3D measurement method based on RGB color fringe projection

Metal parts with highly dynamic areas often appear in industrial production measurements. However, if the traditional fringe projection technique is used to project fringe onto the surface of these metal parts, the light energy will be excessively concentrated and the image will be saturated, resulting thus in the loss of fringe information. To effectively address the high reflectivity problem of the object under test in fringe projection, background normalized Fourier transform contouring was combined with adaptive fringe projection in this work and a new method for performing highly dynamic 3D measurements was proposed. To reduce the number of the acquired images by the camera, a monochromatic fringe of different frequencies was put into the RGB channel to make color composite fringe, and then a color camera was used to acquire the deformed color composite fringe map. The images acquired by the color camera were then separated into three channels to obtain three deformed stripe maps. The crosstalk was also removed from these three images, and the 3D shape of the object was reconstructed by carrying out Fourier transform contouring with background normalization. From our experiments, it was demonstrated that the root mean square error of the proposed method can reach 0.191 mm, whereas, unlike the traditional methods, the developed method requires four images.

Polynomial color compensation model with neural network solver for color-coded fringe patterns in phase-measuring deflectometry

The total image display/projection and image acquisition time in a projector-based phase-measuring deflectometry (PMD) measurement is reduced when color-coded fringe patterns are used since only a single image of the fringe pattern is sufficient to extract phase information using the phase-stepping method. This benefit is, however, offset by the difficulty in resolving various sources of color intensity errors of the fringe patterns in the camera view, such as uneven illumination, color imbalance, color crosstalk, and deviation of color intensity response of the camera and the projector from the calibrated color targets. Thus, a polynomial color compensation model is proposed in this study to compensate for color intensity errors. The model coefficients were estimated using the relationship between the captured and targeted color intensities, which are sequences of monochromatic 8-bit red, green, and blue (RGB) images for projection. The model was applied to modify the color intensities of the generated fringe patterns before projection. The modified color fringe patterns were then used for PMD measurement on spherical concave and convex mirrors. Based on the experimental results, the root-mean-squared (RMS) color intensity errors in R, G, and B channels were reduced by 33.3%, 58.3%, and 36.4%, respectively, after conducting the polynomial color compensation. The RMS phase errors in the fringe patterns were reduced by 71.9%. The RMS height errors and the radius percentage errors of the reconstructed concave and convex heightmaps were 1.87 μm (0.12%) and 1.55 μm (0.17%), respectively. The performance of the proposed polynomial color compensation model in reducing color intensity errors of the captured fringe patterns for accurate PMD measurement was demonstrated successfully.

Isochromatic-Art: A Computational Dataset for Digital Photoelasticity Studies

The importance of evaluating the stress field of loaded structures lies in the need for identifying the forces which make them fail, redesigning their geometry to increase the mechanical resistance, or characterizing unstressed regions to remove material. In such work line, digital photoelasticity highlights with the possibility of revealing the stress information through isochromatic color fringes, and quantifying it through inverse problem strategies. However, the absence of public data with a high variety of spatial fringe distribution has limited developing new proposals which generalize the stress evaluation in a wider variety of industrial applications. This dataset shares a variated collection of stress maps and their respective representation in color fringe patterns. In this case, the data were generated following a computational strategy that emulates the circular polariscope in dark field, but assuming stress surfaces and patches derived from analytical stress models, 3D reconstructions, saliency maps, and superpositions of Gaussian surfaces. In total, two sets of ‘101430’ raw images were separately generated for stress maps and isochromatic color fringes, respectively. This dataset can be valuable for researchers interested in characterizing the mechanical response in loaded models, engineers in computer science interested in modeling inverse problems, and scientists who work in physical phenomena such as 3D reconstruction in visible light, bubble analysis, oil surfaces, and film thickness.

Adaptive Phase Error Suppression Concerning 3D surface Deformation Measurement on Color Digital Fringe Projection Profilometry

3D surface measurement based on phase-shifting profilometry (PSP) has been actively developed in recent years. Three color channels of RGB that are modulated to generate a one-shot PSP method is a concept of color digital fringe pattern profilometry (CDFPP). The CDFPP is a promising technique for the 3D imaging profile of dynamic surface deformation if several phase errors in the one-shot PSP method can be suppressed. This work proposes a processing scheme for phase error suppression schemes (PESS) based on retrieving the modulated sinusoidal fringe and color fringe normalization in PSP using RGB color channel. The processing of PESS consists of tunable bandpass filtering (BPF) followed by fringe normalization. The initial BPF function is defined based on a smoothing spline data set of frequency and power spectrum from the baseline color fringe image. The predefine BPF function could be tunable during the imaging process by considering each frame's condition and RGB channel spectrum mapping. The corrected fringe images are then normalized from the color imbalance, and the phase shift is calculated using the conventional three-step PSP. For evaluation, PESS is performed to reconstruct simulator membrane deformation from four different static profiles and tested to observe the 3D surface of continuous membrane deformation. The PESS could suppress the phase errors of less than 30% less absolute errors than the conventional method and successfully reconstruct the 3D surface for low-frequency continuous membrane deformation with minimizing phase errors.

Phase Target-Based Calibration of Projector Radial Chromatic Aberration for Color Fringe 3D Measurement Systems.

The camera and projector are indispensable hardware parts of a color fringe projection 3D measurement system. Chromatic aberration between different color channels of the projector and camera has an impact on the measurement accuracy of the color fringe projection 3D profile measurement. There are many studies on camera calibration, but the chromatic aberration of the projector remains a question deserving of further investigation. In view of the complex system architecture and theoretical derivation of the traditional projector radial chromatic aberration method, a phase target based on projector radial chromatic aberration measurement and the correction method are proposed in this paper. This method uses a liquid crystal display with a holographic projection film as the phase target. The liquid crystal display sequentially displays red, green, and blue horizontal and vertical sinusoidal fringe images. The projector projects red, green, and blue horizontal and vertical sinusoidal fringe images to the phase target in turn, and calculates the absolute phases of the display fringes and reflection fringes, respectively. Taking the green channel as the reference channel, a phase coordinate system is established based on the phases of the vertical and horizontal directions displayed on the display screen, using the phase of the reflection fringes on the display screen as the ideal phase value of the phase point. Then, the phase coordinate system of the red and blue channels is transferred to the green phase coordinate system to calculate the chromatic aberration of the red-green channels and the blue-green channels, and pre-compensation is conducted. Experimental results prove that this method can measure and calibrate the radial chromatic aberration of the projector without being affected by the image quality of the camera. The correction effect of this method is that the maximum chromatic aberration of the red-green channel decreases from 1.9591/pixel to 0.5759/pixel, and the average chromatic aberration decreases from 0.2555/pixel to 0.1865/pixel. In addition, blue-green channel maximum chromatic aberration decreased from 1.8906/pixel to 0.5938/pixel, and the average chromatic aberration decreased from 0.2347/pixel to 0.1907/pixel. This method can improve the projection quality for fringe projection 3D profile measurement technology.

Single frame color fringe projection profilometry based on empirical mode decomposition imbalance correction algorithm for fast three-dimensional reconstruction

Phase-shifting profilometry (PSP) is considered to be the most accurate technique for phase retrieval with fringe projection profilometry (FPP) systems. However, PSP requires that multiple phase-shifted fringe patterns be acquired, usually sequentially, which limits PSP to static or quasistatic imaging. We introduce color FPP (CFPP), which provides three-dimensional (3D) imaging using a single acquisition, for the application of real-time profiling. A single frame acquisition provides all three phase-shifted fringe patterns needed for the PSP phase retrieval algorithm. However, some problems must be overcome in CFPP, namely, color coupling and imbalance. Following up on the study, we propose a fixed coefficient decoupling method based on point-to-point correction and then introduce the empirical mode decomposition algorithm to decompose a color fringe pattern into three high-frequency components and three low-frequency components for imbalance correction, which improves measurement accuracy and realizes a single frame measurement for complex and discontinuous objects. After correction by the proposed method, the shape of the tested object can be recovered by one a single frame color fringe pattern, which meets the need for dynamic measurement, and the accuracy is not limited by the movement speed of object. The simulations and experimental results show that the proposed method can significantly reduce the influence of color coupling and imbalance in CFPP, and thus realize dynamic and high-precision 3D reconstruction.

Rapid 3D measurement of colour objects based on three-channel sinusoidal fringe projection

A rapid 3D measurement method based on three-channel projection is proposed in this paper. Project a composite color fringe map and an all-white map to the object to be measured. Firstly, crosstalk correction is performed on the composite color fringe pattern collected by the camera then the reflectivity of fringe image is corrected by all-white image. Then, the background normalization principle is used to solve the spectrum aliasing problem when the Fourier transform obtains a truncated phase. Finally, the three-frequency heterodyne method is used for phase unwrapping to obtain the continuous phase and 3D shape. Compared with the traditional color fringe projection method, this method not only eliminates the spectrum aliasing problem but also solves the uneven fringe reflectivity problem when measuring color objects using an all-white map. This method only requires two images to achieve measurement of 3D color dynamic objects.

Single-shot high-precision 3D reconstruction with color fringe projection profilometry based BP neural network

In order to reduce the influence of nonlinear factors such as gamma nonlinearity and color crosstalk on the phase retrieval accuracy with color fringe projection profilometry (CFPP) in rapid measurement, a single-shot high-precision backpropagation (BP) neural network based CFPP is proposed. Firstly, neural networks are trained from samples through a training process. The neural network inputs are the RGB components of composite color fringe patterns. The numerator and denominator of the arctangent function are used as network outputs. The monochromatic fringe patterns are adopted. The numerator and denominator of the arctangent function are calculated by the 15-step phase-shift method, which serves as the network training target. Applying this neural network to the prediction of the standard phase can reduce the influence of nonlinear factors on the phase accuracy. Then, the absolute phase was unwrapped by using geometric constraints. Finally, the 3D reconstruction with high precision can be realized by using only a single-shot color fringe. The theoretical model of the proposed method and the process of neural network training are introduced. The results of 3D reconstruction are compared with other algorithms in the same condition. The experimental results demonstrate that the RMS of the proposed algorithm can reach 0.0107 mm, compared with 0.0121 mm of the traditional 3-step phase-shift method. The proposed method is applied to the online quality inspection of relay components, and the results verify the high accuracy and high efficiency of measurement.

Intensity-Averaged Double Three-Step Phase-Shifting Algorithm with Color-Encoded Fringe Projection

Fringe projection profilometry (FPP) has been broadly employed for three-dimensional shape measurements. However, the measurement accuracy suffers from gamma nonlinearity. This paper proposes an intensity-averaged double three-step phase-shifting (IDTP) algorithm making use of color-encoded fringe projection, which does not require complex calibration processes or extra fringe patterns. Specifically, two phase maps with π/2 phase shift are encoded into the red and blue channels of color fringe patterns. The average fringe patterns of the red and blue channels are approximately in sinusoidal waveform with little harmonics, thus can be directly used for accurate phase recovery. Additionally, an adaptive weight is also estimated for average operation to suppress the effect of color crosstalk. Both simulations and experiments demonstrate that the proposed IDTP algorithm can effectively eliminate nonlinear phase errors.

PhotoelastNet: a deep convolutional neural network for evaluating the stress field by using a single color photoelasticity image.

Quantifying the stress field induced into a piece when it is loaded is important for engineering areas since it allows the possibility to characterize mechanical behaviors and fails caused by stress. For this task, digital photoelasticity has been highlighted by its visual capability of representing the stress information through images with isochromatic fringe patterns. Unfortunately, demodulating such fringes remains a complicated process that, in some cases, depends on several acquisitions, e.g., pixel-by-pixel comparisons, dynamic conditions of load applications, inconsistence corrections, dependence of users, fringe unwrapping processes, etc. Under these drawbacks and taking advantage of the power results reported on deep learning, such as the fringe unwrapping process, this paper develops a deep convolutional neural network for recovering the stress field wrapped into color fringe patterns acquired through digital photoelasticity studies. Our model relies on an untrained convolutional neural network to accurately demodulate the stress maps by inputting only one single photoelasticity image. We demonstrate that the proposed method faithfully recovers the stress field of complex fringe distributions on simulated images with an averaged performance of 92.41% according to the SSIM metric. With this, experimental cases of a disk and ring under compression were evaluated, achieving an averaged performance of 85% in the SSIM metric. These results, on the one hand, are in concordance with new tendencies in the optic community to deal with complicated problems through machine-learning strategies; on the other hand, it creates a new perspective in digital photoelasticity toward demodulating the stress field for a wider quantity of fringe distributions by requiring one single acquisition.

All-day thin-lens computational imaging with scene-specific learning recovery.

Modern imaging optics ensures high-quality photography at the cost of a complex optical form factor that deviates from the portability. The drastic development of image processing algorithms, especially advanced neural networks, shows great promise to use thin optics but still faces the challenges of residual artifacts and chromatic aberration. In this work, we investigate photorealistic thin-lens imaging that paves the way to actual applications by exploring several fine-tunes. Notably, to meet all-day photography demands, we develop a scene-specific generative-adversarial-network-based learning strategy and develop an integral automatic acquisition and processing pipeline. Color fringe artifacts are reduced by implementing a chromatic aberration pre-correction trick. Our method outperforms existing thin-lens imaging work with better visual perception and excels in both normal-light and low-light scenarios.