Fringe Projection System Research Articles

A flexible lateral chromatic aberration correction method for a color-encoded fringe projection system

A flexible correction method is proposed for improving the performance of a color-encoded fringe projection system which suffers from lateral chromatic aberration (LCA) effects. The method uses a white plate with discrete markers and orthogonal fringe patterns projected on its surface to establish the relationships between depth information and pixel deviations among three color channels, and then corrects color-encoded fringe patterns affected by LCA with the acquired relationship for three-dimensional (3D) reconstruction. Compared with the methods that use a translation stage to accurately locate the plate to several parallel positions, the proposed method is flexible because the plate can be placed at arbitrary positions and directions in the measuring volume without an extra translation stage. Two experiments, including 3D shape measurement of a white plate and 3D shape measurement of a regular spherical object are carried out to verify the effectiveness and accuracy of the proposed method.

Evaluating parametric uncertainty using non-linear regression in fringe projection

Optical coordinate measurement systems, such as fringe projection systems, offer fast, high-density measurements of arbitrary surface topographies. The versatility, speed and information density of fringe projection measurements make them attractive as in-situ measurement devices and autonomous inspection systems. However, the complex nature of the measurement process makes evaluating uncertainty from a fringe projection measurement complex – even in the hypothetical simple case where the accuracy of a measurement is limited only by the accuracy in the quantities that define a measurement from an indication; named system parameters here. In this paper, by validating a series of assumptions, a method to explore the uncertainty in the system parameters of a fringe projection system is given. The results of this investigation imply the common distortion model (the Brown-Conrady model) is not specific enough to the camera or projector of a fringe projection system to evaluate its uncertainty.

Warpage characterization of a large size fan-out panel subjected to inhomogeneous heating by a digital fringe projection system

Warpage is an essential issue of fan-out wafer/panel level packaging. Increasing size of wafers/panels poses new challenges to the warpage characterization system. In this work, a warpage measurement system based on digital fringe projection method is reported. It can cover 600 mm × 600 mm area with 4 × 4 camera array. Benchmark tests were carried out to evaluate the measurement error, which was found to be 0.053 %. The warpage evolution of a 515 mm × 510 mm panel with inhomogeneous heating was then studied by heating from room temperature to 90 °C. It was found that the warpage evolution of a large size panel can be very complex. Based on finite element modeling results, this can be attributed to temperature gradient, and several other factors, including mismatch of coefficient of thermal expansion, gravity, initial stress state and original layout design of the panel (such as size and count of chips, RDL design and chip position).

Efficient freeform-based pattern projection system for 3D measurements.

For three-dimensional (3D) measurement of object surface and shape by pattern projection systems, we used a hybrid projection system, i.e., a combination of a projection lens and a transmissive freeform to generate an aperiodic sinusoidal fringe pattern. Such a freeform effects a light redistribution, thus leading to an effective and low-loss pattern projection, as it increases the total transmission intensity of the system and has less power dissipation than classical projection systems. In this paper, we present the conception and realization of the measurement setup of a transmissive fringe projection system. We compare the characteristics of the generated intensity distribution with the classical system based on GOBO (GOes Before Optics) projection and show measurement results of different surface shapes, recorded with the new system.

Three-dimensional reconstruction from a fringe projection system through a planar transparent medium.

A vision measurement system is placed in a protective cover made of a transparent medium to avoid environmental influences. Due to the deflection of light rays on the front and rear surfaces of the transparent medium, the imaging position of an object on the camera target plane is deviated, which makes the traditional vision detection methods based on the triangulation principle produce large measurement errors. This work introduces a three-dimensional (3D) reconstruction method by fringe projection system through a planar transparent medium. We derive the coordinate transformation relationship between a real-object point and a pseudo-object point caused by light refraction based on Snell's law of flat refraction. Based on the relationship, a modified fringe projection method is proposed for unbiased 3D reconstruction. Two experiments, including 3D shape measurement of a white plate with ring markers and 3D shape measurement of a regular spherical object are conducted. The results demonstrate the effectiveness of the proposed method in such measurement environment.

Motion-Induced Phase Error Compensation Using Three-Stream Neural Networks

Phase-shifting profilometry (PSP) has been widely used in the measurement of dynamic scenes. However, the object motion will cause a periodical motion-induced error in the phase map, and there is still a challenge to eliminate it. In this paper, we propose a method based on three-stream neural networks to reduce the motion-induced error, while a general dataset establishment method for dynamic scenes is presented to complete three-dimensional (3D) shape measurement in a virtual fringe projection system. The numerous automatically generated data with various motion types is employed to optimize models. Three-step phase-shift fringe patterns captured along a time axis are divided into three groups and processed by trained three-stream neural networks to produce an accurate phase map. The actual experiment’s results demonstrate that the proposed method can significantly perform motion-induced error compensation and achieve about 90% improvement compared with the traditional three-step phase-shifting algorithm. Benefiting from the robust learning-based technique and convenient digital simulation, our method does not require empirical parameters or complex data collection, which are promising for high-speed 3D measurement.

Dual-biprism-based coaxial fringe projection system

Fringe projection profilometry (FPP) has been widely used for three-dimensional shape measurement because of its simple hardware and high measurement accuracy. However, the use of most FPP systems to measure complex surfaces such as stepped height objects can result in severe occlusion and shadow. To alleviate this problem, a coaxial measurement method based on dual biprisms is presented in this paper. The system is low-cost and easy to implement, and does not require the introduction of mechanical moving parts. Only dual biprisms are introduced into the coaxial system to provide the geometric constraints required for reconstruction. Experimental results on the reconstruction of a surface with significant height changes demonstrate that the proposed method can achieve shadow-free measurements.

Metrology for Measuring Bumps in a Protection Layer Based on Phase Shifting Fringe Projection

We developed a telecentric 3D measurement system based on the fringe projection technique to measure phases corresponding to the 3D shape of bumps on a surface. A measurement algorithm based on an area scan camera with a flexibly programmable region of interest (ROI) was applied to reconstruct the 3D model. Only the pixel information within the ROI is stored, which reduces the data volume and processing and accelerates the area scan acquisition. Experiments showed that the fringe projection system provided good linearity between the calibration and optical data with excellent accuracy. A reflectometry sensor was implemented to provide a correction offset to the bump height in a protection layer in a subsequent high-speed inspection.

Simultaneous intensity projection for projector response computation with application to an active radiometric calibration

One of the most important sources of errors in digital fringe projection (DFP) systems is the nonlinearity in the response of the projector when it uses the three-step phase retrieval algorithm. Thus, it is necessary to increase the accuracy without affecting the efficiency. In this sense, the radiometric rectification methods are used. In this paper, an active radiometric rectification method for digital fringe projection is proposed. This proposal consists in two improvements of traditional active techniques: first, parallel intensity projection is used to obtain the projector response which requires only four dot patterns; and second, a method is provided for the calculation of the inverse polynomial that guarantees symmetry with respect to the response of the projector. Experimental results, in a digital fringe projection system, show that the root-mean-square phase error improves 6.3 times using this proposal.

Study on Algorithms for the Virtual Assembly and Best Combinations of In-line Measured Injection-Molded Parts

Geometrical deviations of manufactured parts affect the quality of the resulting product. In an assembly, deviations accumulate so that the functional requirements may no longer be met. Therefore, the compliance of geometrical interfaces is of great importance in order to achieve a high-quality product. Modern 3D sensors capture dense point clouds of manufactured parts and thus enable creating a part’s geometrical digital twin. However, there are no suitable registration techniques that allow computing the assembly of the measured parts. With the knowledge of the assembly configuration, the geometrical accumulation can be determined, ultimately allowing the prediction of the quality of the final product. In the authors preceding work, the virtual assembly registration technique was presented methodologically. This paper focusses on the implementation of a dedicated use case. As real-world problem, the assembly of injection-molded covers and housings is examined. The data acquisition is fully integrated into an in-line measurement process. Therefore, a fringe projection system is used for capturing 3D data. Specific segmentation approaches are presented in order to detect the part and to segment assembly-relevant geometries in the measurement data. Due to possible artefacts in the point clouds, strategies for increasing the robustness are discussed. Finally, the developed quality prediction is embedded in the production context. In order to achieve time, cost, and quality benefits, a selective assembly strategy is applied that reduces the number of scrap parts by a factor of two and more.

Ultra-high speed holographic shape and displacement measurements in the hearing sciences

The auditory system of mammals enables the perception of sound from our surrounding world. Containing some of the smallest bones in the body, the ear transduces complex acoustic signals with high-temporal sensitivity to complex mechanical vibrations with magnitudes as small as tens of picometers. Measurements of the shape and acoustically induced motions of different components of the ear are essential if we are to expand our understanding of hearing mechanisms, and also provide quantitative information for the development of numerical ear models that can be used to improve hearing protection, clinical diagnosis, and repair of damaged or diseased ears.We are developing digital holographic methods and instrumentation using an ultra-high speed camera to measure shape and acoustically-induced motions in the middle ear. Specifically we study the eardrum, the first structure of the middle ear which initializes the acoustic-mechanical transduction of sound for hearing. Our measurement system is capable of performing holographic measurement at rates up to 2.1 M frames per second. Two shape measurement modalities had previously been implemented into our holographic systems: (1) a multi-wavelength method with a wavelength tunable laser; and (2) a multi-angle illumination method with a single wavelength laser. In this paper, we present a third method using a miniaturized fringe projection system with a microelectromechanical system (MEMS) mirror. Further, we optimize the processing of large data sets of holographic displacement measurements using a vectorized Pearson's correlation algorithm. We validate and compare the shape and displacement measurements of our methodologies using a National Institute of Standards and Technology (NIST) traceable gauge and sound-activated latex membranes and human eardrums.

Reference Measurement of Roundwood by Fringe Projection

Abstract The metrological verification of log scanners requires logs with accurately known dimensions as test objects. The measurement of the lengths and diameters must be traceable back to the SI (International System of Units) unit of length. The results have to be reported with the corresponding measurement uncertainties. The uncertainties are required to be 5 to 10 times lower than the corresponding maximum permissible errors allowed for the log scanner under test. This article presents a procedure for the reference measurement of logs using an off-the-shelf fringe projection system along with uncertainty budgets for the measured dimensions. The length and diameters are determined from the highly resolved mesh obtained by fringe projection using techniques from computational geometry and coordinate metrology. Corrections are applied to the length and diameter values to remove the systematic effect caused by scattering of projected light below the partially transparent log surface. The influence of the fringe projection system on the measured dimensions is determined by measurements of calibrated artifacts, which also provide the traceability back to the SI unit of length. The measurement is illustrated by the example of a log with a length of 2 m and a diameter of 280 mm. The corresponding uncertainty budgets, confirmed by repeat measurements, result in expanded uncertainties (confidence interval 95%) of 6 mm and 0.13 mm for length and diameter, respectively. These low values qualify the fringe projection measurement along with accompanying evaluation procedure to provide logs as reference objects for the verification of log scanners.

Phase Error Analysis and Correction for Crossed-Grating Phase-Shifting Profilometry.

Crossed-grating phase-shifting profilometry (CGPSP) has great utility in three-dimensional shape measurement due to its ability to acquire horizontal and vertical phase maps in a single measurement. However, CGPSP is extremely sensitive to the non-linearity effect of a digital fringe projection system, which is not studied in depth yet. In this paper, a mathematical model is established to analyze the phase error caused by the non-linearity effect. Subsequently, two methods used to eliminate the non-linearity error are discussed in detail. To be specific, a double five-step algorithm based on the mathematical model is proposed to passively suppress the second non-linearity. Furthermore, a precoding gamma correction method based on probability distribution function is introduced to actively attenuate the non-linearity of the captured crossed fringe. The comparison results show that the active gamma correction method requires less fringe patterns and can more effectively reduce the non-linearity error compared with the passive method. Finally, employing CGPSP with gamma correction, a faster and reliable inverse pattern projection is realized with less fringe patterns.

A Simple Calibration Method for a Fringe Projection System Embedded within an Additive Manufacturing Machine

In additive manufacturing (AM), especially for advanced powder fusion machines, it is of high importance to develop an in situ inspection system to monitor the printed surface and pre-print powder bed as the build cycle proceeds. Consequently, high resolution, high precision and fast detection measurement systems need to be investigated, as such optically based measurement systems can provide feedback for manufacturing process optimisation. Fringe projection technology has a great advantage in the measurement of topography in such environments. The implementation of a fringe projection system requires that the system is pre-calibrated in order to obtain high measurement resolution and repeatability. This paper presents a simple calibration method for an AM-based in situ fringe projection system using a phase-depth calibration model. If a calibration plate with certificated marks is used, however, the texture of the plate will affect the measured phase accuracy. A simple calibration method to reduce the calibration plate texture effect in the process of calibration is outlined. Experimental results show that the proposed method can eliminated these effects and improve measurement resolution and repeatability. The proposed in situ/in process inspection technique has been implemented within a commercial electron beam powder bed fusion additive manufacturing machine (EBAM), to demonstrate the capability for effective feedback during the manufacturing process.

Enhanced geometric constraint-based phase unwrapping algorithm in binocular stereo vision fringe projection system

Phase unwrapping plays an important and central role in phase-based digital fringe projection profilometry. The unwrapping quality directly influences the three-dimensional measurement accuracy. Recently, an effective geometric constraint-based phase unwrapping algorithm has been proposed to obtain the continuous absolute phase map and the unwrapped phase accuracy was found to be high. However, in this technique the virtual depth plane at z = zmin is often created empirically, which increases the manual measurement error. For this reason, this paper proposes a method for accurately constructing the virtual plane and further applies it to phase unwrapping of objects with a larger depth range. In this method, a binocular stereo vision system is used as the measurement set-up for the virtual depth plane construction and a series of virtual depth planes at z = zimin (i ≥ 2) is automatically built using a computational framework. Then, the phase is unwrapped for each region according to the continuity of the unwrapped phase and a complete absolute phase map is obtained by merging the unwrapped phases in all regions for 3D reconstruction. In this process, the virtual depth planes are created automatically and quantitatively by the measurement system. No human intervention is required and it greatly reduces the manual measurement error. Experiments show that the artificial virtual planes can be built accurately and the phase is unwrapped correctly and readily.

High-dynamic-range 3D measurement for E-beam fusion additive manufacturing based on SVM intelligent fringe projection system

Electron Beam Metal Additive Manufacturing (EBAM) has been developed over recent years because of its advantages in manufacturing internal features and complex structures with relatively high productivity. The process proceeds by layer by layer melting and re-solidification of metal powder utilising an electron beam energy source. Following solidification of the build layer, the surface becomes ‘shiny’ with high reflectivity which makes in-process inspection of the surface of the build layer using fringe projection, difficult. To address this issue, a novel intelligent fringe projection technique using a support-vector-machine (SVM) algorithm is proposed to measure the 3D topography of high dynamic range surfaces on a layer by layer basis within the EBAM machine. To facilitate the SVM implementation a range of EBAM manufactured surfaces are utilised as samples for training and classification. The training measurements are based on different exposure times and saturated pixels are utilised as feature vectors to predict tested samples categories. Training errors are evaluated, and the correct recognition rate is 91%, which indicates the proposed training method can effectively predict the categories of the surfaces. Examples of melting edge swelling and powder bed inspections during a part build are used to demonstrate the system capability for inspection high dynamic range measurement within the EBAM machine. The whole inspection process lasts less than 5 s with two measurements which minimises the time penalty for the manufacturing process. Experimental results showed that the powder and the melting surface defects could be efficiently inspected using the proposed technology and the measurement result could be fed back to the build process to improve the processing quality.

Efficient empirical determination of maximum permissible error in coordinate metrology

Maximum permissible errors (MPEs) are an important measurement system specification and form the basis of periodic verification of a measurement system’s performance. However, there is no standard methodology for determining MPEs, so when they are not provided, or not suitable for the measurement procedure performed, it is unclear how to generate an appropriate value with which to verify the system. Whilst a simple approach might be to take many measurements of a calibrated artefact and then use the maximum observed error as the MPE, this method requires a large number of repeat measurements for high confidence in the calculated MPE. Here, we present a statistical method of MPE determination, capable of providing MPEs with high confidence and minimum data collection. The method is presented with 1000 synthetic experiments and is shown to determine an overestimated MPE within 10% of an analytically true value in 99.2% of experiments, while underestimating the MPE with respect to the analytically true value in 0.8% of experiments (overestimating the value, on average, by 1.24%). The method is then applied to a real test case (probing form error for a commercial fringe projection system), where the efficiently determined MPE is overestimated by 0.3% with respect to an MPE determined using an arbitrarily chosen large number of measurements.

Fringe Projection Profilometry in Production Metrology: A Multi-Scale Comparison in Sheet-Bulk Metal Forming.

Fringe projection profilometry in combination with other optical measuring technologies has established itself over the last decades as an essential complement to conventional, tactile measuring devices. The non-contact, holistic reconstruction of complex geometries within fractions of a second in conjunction with the lightweight and transportable sensor design open up many fields of application in production metrology. Furthermore, triangulation-based measuring principles feature good scalability, which has led to 3D scanners for various scale ranges. Innovative and modern production processes, such as sheet-bulk metal forming, thus, utilize fringe projection profilometry in many respects to monitor the process, quantify possible wear and improve production technology. Therefore, it is essential to identify the appropriate 3D scanner for each application and to properly evaluate the acquired data. Through precise knowledge of the measurement volume and the relative uncertainty with respect to the specimen and scanner position, adapted measurement strategies and integrated production concepts can be realized. Although there are extensive industrial standards and guidelines for the quantification of sensor performance, evaluation and tolerancing is mainly global and can, therefore, neither provide assistance in the correct, application-specific positioning and alignment of the sensor nor reflect the local characteristics within the measuring volume. Therefore, this article compares fringe projection systems across various scale ranges by positioning and scanning a calibrated sphere in a high resolution grid.

The instrument transfer function for optical measurements of surface topography

For optical measurements of areal surface topography, the instrument transfer function (ITF) quantifies height response as a function of the lateral spatial frequency content of the surface. The ITF is used widely for optical full-field instruments such as Fizeau interferometers, confocal microscopes, interference microscopes, and fringe projection systems as a more complete way to characterize lateral resolving power than a single number such as the Abbe limit. This paper is a comprehensive review of the ITF, including standardized definitions, ITF prediction using theoretical simulations, common uses, limitations, and evaluation techniques using material measures.

Characterisation of a multi-view fringe projection system based on the stereo matching of rectified phase maps

Multi-view fringe projection systems can be effective solutions to address the limitations imposed by the limited field of view, line-of-sight issues and occlusions when measuring the geometry of complex objects, associated with single camera–projector systems. However, characterisation of a multi-view system is challenging since it requires the cameras and projectors to be in a common global coordinate system. We present a method for characterising a multi-view fringe projection system which does not require the characterisation of the projector. The novelty of the method lies in determining the correspondences in the phase domain using the rectified unwrapped phase maps and triangulating the matched phase values to reconstruct the three-dimensional shape of the object. A benefit of the method is that it does not require registration of the point clouds acquired from multiple perspectives. The proposed method is validated by experiment and comparison with a conventional system and a contact coordinate measuring machine.