3D Shape Measurement Research Articles

Motion-induced phase shift for dynamic structured light measurement.

Structured light 3D shape measurement is extensively utilized in semiconductor inspection, smart manufacturing, and biomedical imaging due to its rapid measurement speed, high precision, and versatile applicability to different objects. However, the traditional implementations of this method often require that the object remains static while recording the phase-shifting structured light images, which limits the adaptability of dynamic measurement. Here, we propose a dynamic 3D shape measurement using structured light based on a motion-induced phase shift (MIPS). As the object moves, the surface features distort the fringe pattern, resulting in a phase-shifting effect. By employing the MIPS method, we can determine the phase even in the situations where the knowledge of phase-shifting conditions is not accurate. This enables the acquisition of the 3D topography of the object surface with a high level of precision. Experimental results demonstrate that the MIPS method can accurately measure the 3D shape of objects moving as fast as 100 mm/s, with a relative discrepancy of less than 0.23%.

Digital Manufacturing System of Custom Wigs with Measurable 3D Images

Wigs are major important means for improving life quality of the increasing people suffered from hair loss through their image enhancements. For the production of custom made wigs, there requires a mold that reproduces the wearer’s head dimension and the shape of hair loss. The conventional method for this is to make a resin film that adheres to the wig wearer’s head using vinyl tape, and to make a wig pattern with drawing the shape of hair loss onto the film, and to produce a head mold with this pattern, and then to plant hair in a wig cap made based on this mold and wig pattern. This method requires highly skilled manual work, and generates a large amount of chemical waste during the manufacturing process, and takes a long manufacturing period. However, the digital automation of measuring head dimension and the shape of hair loss can replace or omit the production of wig patterns. In this study, a 3D head image capable of actual measurement was implemented by using a 3D polygon head shape measurement device and a camera, which makes a digital production system possible, which can replace the manual wig pattern production process for custom wigs.

3D shape measurement method for high-reflective surface based on a color camera

Fringe projection profilometry (FPP) has been widely utilized in many fields due to its non-contact, high accuracy, high resolution and full-field measurement capabilities. However, the limited dynamic range of the camera sensor can result in overexposure of high-reflective parts in industrial production measurement. To effectively solve the above issue, this paper proposes a 3D shape measurement method for the high-reflective surface based on a color camera. Firstly, the optimal exposure time for the low-reflective region is estimated using the relationship between the standard variance of the phase error and the modulation. Twelve blue phase-shifted fringe patterns and a uniform blue image are utilized to obtain 3D shape of high-reflective surface. Secondly, captured images are separated into red, green and blue components. The fused Gaussian low-pass filtering method with different cut-off frequencies is used to filter and denoise the green or red components and the true intensity of the blue component in the saturated regions is calculated by the unsaturated intensity of the green or red components based on the calibrated crosstalk matrix. Then the image in saturated regions is fused and normalized with the unsaturated region. The absolute phase obtained from the fused normalized fringe patterns is converted into 3D data. Finally, experiments have been carried out on measuring. The experimental results show that the proposed method is capable of obtaining the 3D shape of the surface of a high-reflective object with fewer patterns and high measurement accuracy.

Brightness and contrast corrections for space–time stereocorrelation via proper generalized decomposition

Stereocorrelation (SC) is a flexible, non-contact experimental tool for measuring 3D shape and surface kinematics in complex mechanical tests. The violation of gray level conservation raises the issue of convergence of SC (e.g., when local specular reflections are observed). Brightness and contrast corrections (BCCs) then become necessary. Space–time separation allows modes of the displacement, brightness, and contrast fields to be computed on the fly. The paper introduces a framework for Brightness and Contrast Corrections in Proper Generalized Decomposition stereocorrelation (BCCs-PGD-SC). It is shown that BCCs-PGD-SC improves the faithfulness of measured displacement fields and makes the identification of cracks viable from residual fields. The computational cost to include BCCs in PGD stereocorrelation is shown to be minimal within a PGD framework.

Structured light 3D shape measurement for translucent media base on deep Bayesian inference

Traditional structured light technique faces challenges in measuring translucent media due to low fringe modulation and strong random noise caused by subsurface scattering, thereby significantly reducing phase quality. In addition, the difficulty in obtaining ground truth makes it hard to assess reliability even though obtaining measured results. Here, we proposed a 3D measurement method for translucent media base on deep Bayesian inference to achieve both fringe enhancement and phase uncertainty evaluation. Specifically, a deep network incorporated with quatuor-branch residual block is developed to significantly enhance the fringe modulation and signal-to-noise ratio (SNR) for accurate phase recovery. Subsequently, a Bayesian inference mechanism is established for probabilistic statistics, which allows for the optimization of fringe output and provides uncertainty self-evaluation based on Monte Carlo (MC) sampling. Furthermore, by incorporating both numerical and physical constraints into the supervised learning, the network can effectively mitigate phase-shifted errors in the final results. The proposed method shows high efficiency and flexibility since it requires no additional patterns or hardware setup. Experiments validate the feasibility of the proposed method.

Ultra high-speed 3D shape measurement technology for specular surfaces based on μPMD

Phase measuring deflectometry (PMD) has been extensively applied to measure specular surfaces due to its non-contact, high-precision, full-field measurement capabilities. Liquid crystal display (LCD) screen is the most common structured light source in PMD. However, the response time of liquid crystal molecules limits its frame rate to around 100 frames per second (fps). Therefore, it is quite difficult for traditional PMD to measure rapidly moving surfaces. This paper proposes a 3D dynamic sensing technique, microsecond-PMD (µPMD) based on the high-frame-rate sinusoidal fringe display (HSFD). In the proposed method, the switching time for each fringe pattern display is at a sub-microsecond level, enabling high-speed fringe acquisition with kHz-level area array detection or 100kHz-level line array scanning. The HSFD method uses a specially designed LED array and two-step optical expansion. The high-speed switching characteristic of LED sources is utilized to allow a superfast display rate. Moreover, the superior sinusoidal property can be achieved by the combination of the specially designed discrete sinusoidal LED array, the light-diffracting effect of orthogonal gratings, and the filtering effect of the light diffuser. The mechanism and analytic model of fringe generation are thoroughly analyzed and discussed in this work. Furthermore, the swarm optimization algorithm and corresponding weighted fringe quality evaluation function are presented to obtain the optimal fringes. To the best of our knowledge, the proposed µPMD, for the first time, achieved a superfast fringe acquisition rate of 4000fps with sub-micrometer precision in three-dimensional (3D) reconstruction for specular surfaces. We envision this proposal to be broadly implemented for real-time monitoring in manufacturing.

Cranial Remolding Orthosis Study on the Use of a Temperature Sensor to Measure Wear Time

ABSTRACT Introduction The effectiveness of cranial remolding orthoses (CROs) in treating nonsynostotic deformational plagiocephaly (DP) is anecdotally related to adherence. Efficacy of CROs has been evaluated but lacks evidence regarding necessary wear time to achieve a positive outcome. A 23 hr/d wear schedule is generally prescribed regardless of presentation. The study compares daily wear time to treatment outcomes. Materials and Methods Subjects Infants aged 3–18 months diagnosed with DP were included with initial cranial vault asymmetry (CVA) > 6 mm or cephalic ratio (CR) > 0.90. Subjects were treated with an Orthomerica STARband CRO. A total of 106 subjects enrolled, with 69 completed. Apparatus A questionnaire assessed caregiver’s reported adherence with the CRO. Maximum Integrated’s iButton temperature loggers (iButtons) recorded objective wear time. Procedures 3D head shape measurements via Orthomerica STARScanner, caregivers’ questionnaires, and iButton data were collected every 5–8 weeks. Data Analysis Descriptive statistics were obtained, and data reported as mean ± SD or median [25th, 75th percentiles]. Nonparametric sign tests were used to assess differences from CRO fabrication scan to subsequent follow-ups. Spearman rank correlations and corresponding 95% confidence intervals and P values between average wear time and change of measurements were obtained. Results Self-reported wear time was 22 (22, 23) hrs/d. Measured wear time was 17–18 (12, 20) hrs/d. Longer average wear time was significantly associated with larger reductions in CVA (P = 0.0054), CR (P = 0.0080), and CVAI (P = 0.0059). Conclusions Results showed longer average daily wear increased effectiveness in CRO treatment of plagiocephaly. Increased sample sizing is required to determine if generalizable to brachycephalic and asymmetrical brachycephalic head shapes. Clinical Relevance A CRO wear schedule of 23 hrs a day is currently recommended, but difficult for families to adhere. The impact of the study may indicate a more realistic wear schedule that would improve adherence while achieving optimal outcomes.

Spatial phase unwrapping approach for single-frame 3D shape measurement based on deep learning

To address the challenge of balancing accuracy and speed in traditional phase unwrapping algorithms, this paper proposes a deep-learning-based single-frame spatial phase unwrapping method. By leveraging extensive data learning, two neural networks are trained to directly acquire phase information and modulation from a single-frame fringe pattern. Then, through the integration of a modulation sorting phase unwrapping algorithm, we achieve high-precision 3D surface reconstruction from a single-frame fringe pattern, thereby enabling rapid object measurement. The experimental results demonstrate the remarkable accuracy of the proposed method in phase unwrapping, approaching the level achieved by the 12-step phase-shifting method. The integration of deep learning into phase unwrapping offers promising prospects for further developments in this area. This advancement holds significant implications for high-speed measurement in the manufacturing field.

3D shape measurement based on Res-Attention-Unet for deep learning

3D shape measurement based on Res-Attention-Unet for deep learning

Optimal multiple fringe pattern composition by weighted complex amplitude fusion in fringe projection profilometry

This paper presents an optimal multiple fringe pattern composition method for 3D shape measurement of high-dynamic-range (HDR) objects using fringe projection profilometry (FPP). With the inverse variance weighting theory, we take the square of the modulation intensities of the fringe pattern images with different intensity levels as the weights to obtain the composited phase of fringe patterns by weighted complex amplitude fusion, which improves the measurement precision of HDR objects. Additionally, we integrate HDR 3D shape measurement and temporal noise reduction into a unified framework by utilizing weighted complex amplitude fusion to completely measure translucent objects with specular reflections. Simulations and experiments demonstrate that our method can achieve higher measurement precision and is resistant to the time-varying ambient light.

Fast high-precision 3D shape measurement using linear phase encoding with geometry constraints

A three-dimensional (3D) profile measurement method based on geometric constraints combined with linear phase encoding (GCPE) is proposed. This method encodes the wrapped phase and the linear signal in the same phase domain, achieves the elimination of the phase ambiguity within the local fringe period, and obtains an unambiguous absolute phase within the entire period through geometric constraints. Compared with the traditional phase encoding method, this method solves the problem of period sequence errors caused by a large number of codewords by encoding linear signals in the phase domain and using a period order correction method to deal with the period jump phenomenon caused by noise. At the same time, the measurement range of the fringe projection system under geometric constraints is significantly improved. Experimentally, 3D profiles of standard planes, complex statues, and separated objects were measured by the use of the GCPE. The results show that the GCPE has the advantage of fast speed and high accuracy in measuring the 3D profile of objects.

Three 1-bit speckle-embedded pulse-width modulation patterns for robust absolute 3D measurement

In three-dimensional (3D) shape measurement techniques using structured light, 1-bit pulse-width modulation (PWM) patterns and 1-bit speckle patterns can be projected at high speed. However, when combining PWM and speckle patterns to integrate their advantages, the decoupling problem is insurmountable. In this work, a novel 1-bit speckle-embedded PWM (SPPWM) method was proposed to achieve absolute 3D shape measurement using only three binary patterns. Our method consists of three main steps: First, a sinusoidal pattern reconstruction network was proposed to eliminate the high-order harmonics and speckle patterns in the SPPWM patterns and obtain high-quality sinusoidal patterns. Second, a multi-temporal spatial correlation matching algorithm was proposed to obtain a coarse disparity map from the three SPPWM patterns. Third, the high-accuracy wrapped phase map is used as an additional constraint for refining the coarse disparity map to obtain the final high-accuracy disparity map for absolute 3D measurement without phase unwrapping. Our method combines the advantages of fringe projection profilometry techniques for high-precision wrapped phase retrieval and speckle correlation matching algorithms for robust and unambiguous disparity map calculation. The experimental results demonstrated that our method could realize high-precision absolute 3D shape measurement with an accuracy of 0.057 mm using only three 1-bit SPPWM patterns. Furthermore, different simulation noises were used to demonstrate the robustness of the proposed method.

Three-dimensional shape measurement technique for hot and shiny forging

For hot and shiny forging, rapid optical three-dimensional (3D) shape measurement can find the defects in time, improve manufacturing efficiency and reduce manufacturing cost. Optical 3D measurement of hot and shiny forging faces following three challenges. (a) Influence of blackbody radiation on the red band of visible light leads to saturation of captured fringe pattern. (b) Infrared band of blackbody radiation causes the camera to heat up. (c) Specular reflection of shiny forging leads to local saturation of captured fringe pattern. To address the above issues, we propose an effective optical 3D measurement scheme for hot and shiny forging in this paper. Firstly, the experiments demonstrate that the influence of blackbody radiation on visible light is concentrated in red light band and has no crosstalk in the blue light band, so we adopt blue light projection and blue light band-pass filter (BBF) to capture clear fringe pattern. Secondly, since the BBF cannot block the near-infrared component, we use an infrared light cutoff filter (ICF) to reduce the thermal effect of blackbody radiation. Finally, we analyze the Fourier transform (FT) spectrum of saturated fringe pattern caused by specular reflection, and propose an efficient Butterworth low-pass filtering (BLPF) method that does not require additional hardware, algorithm processing, and image assistance. The experimental results verify the effectiveness of the proposed scheme.

3D Shape Measurement of Aeroengine Blade Based on Fringe Projection Profilometer Improved by Multi-Layer Concentric Ring Calibration.

The precision requirements for aeroengine blade machining are exceedingly stringent. This study aims to improve the accuracy of existing aeroengine blade measurement methods while achieving comprehensive measurement. Therefore, this study proposes a new concentric ring calibration method and designs a multi-layer concentric ring calibration plate. The effectiveness of this calibration method was verified through actual testing of standard ball gauges. Compared with the checkerboard-grid calibration method, the average deviation of the multilayer concentric ring calibration method for measuring the center distance of the standard sphere is 0.02352, which improves the measurement accuracy by 3-4 times. On the basis of multi-layer concentric ring calibration, this study builds a fringe projection profiler based on the three-frequency twelve-step phase shift method. Compared with the CMM, the average deviation of the blade chord length measured by this solution is 0.064, which meets the measurement index requirements of aeroengine fan blades.

Calibration for endoscopic 3D shape measurement with cone beam projection

We demonstrate a calibration method for endoscopic three-dimensional shape measurement with cone beam projection. In this method, changes in the shape of the optical sectioning profiles are quantified and fitted while scanning a calibration board in the depth direction, using a cubic function. In accuracy tests using a flat plate and a ring reference gauge, the proposed method obtains an accuracy of 0.02 mm in the depth dimension and 0.09 mm in the radial dimension. These results represent 88% and 55% improvements compared to previous analysis. For medical applications, an ear examination simulator was employed, and our measurement results were compared to ground truth data obtained by microfocus X-ray computed tomography. The surface deviation of our method relative to the ground truth data was ±0.36mm during manual operation. A comparison of the measurement results before and after calibration revealed an improvement in the peak agreement with the ground truth data, with the deviation shifting from 0.2 mm to −0.05mm. Our strategy achieves a digital transformation of 3D endoscopy, which would benefit a number of medical fields.

Deep learning-based frequency-multiplexing composite-fringe projection profilometry technique for one-shot 3D shape measurement

The pursuit of precise and efficient 3D shape measurement has long been a focal point within the fringe projection profilometry (FPP) domain. However, achieving precise 3D reconstruction for isolated objects from a single fringe image remains a challenging task in the field. In this paper, a deep learning-based frequency-multiplexing (FM) composite-fringe projection profilometry (DFCP) is proposed, where an end-to-end absolute phase retrieval network (APR-Net) is trained to directly recover the absolute phase map from a FM composite fringe pattern. The obtained absolute phase map exhibits exceptional precision and is devoid of spectrum crosstalk or leakage disturbance typically encountered in traditional FM techniques. APR-Net is intricately crafted, incorporating a nested strategy along with the concept of centralized information interaction. A diverse dataset encompassing various scenarios and free from spectrum aliasing is assembled to guide the training process. A seven-map loss calculation scheme is employed to guide the training process, of which the efficacy is proved through ablation experiments. In the first qualitative experiment, DFCP demonstrates comparable phase accuracy to ground truths with 47 fewer projected images, outperforming other three methods with mean absolute phase errors of 0.0052 rad, 0.1761 rad, 0.0169 rad, and 0.0139 rad. The second qualitative experiment and the quantitative evaluation respectively prove DFCP’s capability in high dynamic range 3D measurement and in precise 3D measurement, with sphere diameter errors of 0.0780 mm and 0.0726 mm, and a spherical centroid distance error of 0.0555 mm.

Shape-aware speckle matching network for cross-domain 3D reconstruction

The learning-based binocular 3D reconstruction method outperforms traditional vision algorithms in terms of precision and efficiency. However, in some cross-domain scenes, the precision of the well-trained network drastically decreases when handling samples in diverse contexts. This paper proposes an end-to-end shape-aware speckle matching network (SSMNet) that combines shape-mask information to achieve improved precision and completeness of disparity calculation in cross-domain applications. The cascade attention mechanism is inserted in the feature extraction stage to concentrate on valuable regions. The shape-aware module is designed to learn additional shape contour information, and multiscale features are integrated simultaneously to construct the cost-volume for the subsequent lightweight 3D aggregation. In addition, instance normalization is adopted to guarantee style migration, and a hybrid loss function is used to supervise the learning process. Furthermore, a high-precision binocular speckle dataset is built, including training and testing sets in different distributions. Extensive quantitative and qualitative experiments demonstrate that SSMNet enhances cross-domain capability and achieves state-of-the-art performance. Measurement precision evaluation illustrates that the proposed method can realize the desired highly precise 3D shape measurement in a real industrial scenario.

Underwater 3D reconstruction based on double N-step orthogonal polarization state phase shift strategy

The nonlinear response of projection equipment and the uneven reflectance of HDR (high dynamic range) object surface pose challenges in underwater 3D reconstruction. As a result, the phase-shifting fringe pattern collected by the camera does not have good sinusoidality, leading to phase errors. Therefore, this paper proposes a double N-step orthogonal polarization state phase-shift strategy (DOPS). This technique uses the degree of linear polarization (DOLP) of polarized light to design a double N-step phase-shifting fringe pattern. The original phase shift fringes are encoded as horizontal polarization fringes, and the additional phase shift fringes are encoded as vertical polarization fringes. Then, the camera captures and extracts the phase information of the two groups of DOLP patterns, which are separately subjected to phase unwrapping. The resulting unwrapped phases are fused to reduce phase error. The experimental results indicate that this method reduces the measurement error by 57 % when dealing with 3D shape measurement of underwater HDR objects, compared to the polarization-encoded phase shift algorithm (PEPS). Additionally, this method is more accurate in measurement accuracy and improves measurement efficiency by 50 % compared to the double N-step phase shift method (DNPS).

Calibration-free structured-light-based 3D scanning system in laparoscope for robotic surgery.

Accurate 3D shape measurement is crucial for surgical support and alignment in robotic surgery systems. Stereo cameras in laparoscopes offer a potential solution; however, their accuracy in stereo image matching diminishes when the target image has few textures. Although stereo matching with deep learning has gained significant attention, supervised learning requires a large dataset of images with depth annotations, which are scarce for laparoscopes. Thus, there is a strong demand to explore alternative methods for depth reconstruction or annotation for laparoscopes. Active stereo techniques are a promising approach for achieving 3D reconstruction without textures. In this study, a 3D shape reconstruction method is proposed using an ultra-small patterned projector attached to a laparoscopic arm to address these issues. The pattern projector emits a structured light with a grid-like pattern that features node-wise modulation for positional encoding. To scan the target object, multiple images are taken while the projector is in motion, and the relative poses of the projector and a camera are auto-calibrated using a differential rendering technique. In the experiment, the proposed method is evaluated by performing 3D reconstruction using images obtained from a surgical robot and comparing the results with a ground-truth shape obtained from X-rayCT.

Adaptive phase retrieval algorithm for local highlight area based on a piecewise sine function.

Phase measuring profilometry (PMP) has been widely used in industries for three-dimensional (3D) shape measurement. However, phase information is often lost due to image saturation results from high-reflection object surfaces, leading to subsequent 3D reconstruction errors. To address the problem, we propose an adaptive phase retrieval algorithm that can accurately fit the sinusoidal fringes damaged by high reflection in the saturated regions to retrieve the lost phase information. Under the proposal, saturated regions are first identified through a minimum error thresholding technique to narrow down regions of interest and so that computation costs are reduced. Then, images with differing exposures are fused to locate peak-valley coordinates of the fitting sinusoidal fringes. And the corresponding values of peak-valley pixels are obtained based on a least squares method. Finally, an adaptive piecewise sine function is constructed to recover the sinusoidal fringe pattern by fitting the pattern intensity distribution. And the existing PMP technology is used to obtain phase information from the retrieved sinusoidal fringes. To apply the developed method, only one (or two) image with different exposure times is needed. Compared with existing methods for measuring reflective objects, the proposed method has the advantages of short operation time, reduced system complexity, and low demand on hardware equipment. The effectiveness of the proposed method is verified through two experiments. The developed methodology provides industry an alternative way to measure high-reflection objects in a wide range of applications.