Three-dimensional Shape Measurement Method Research Articles

Calibration for endoscopic 3D shape measurement with cone beam projection

We demonstrate a calibration method for endoscopic three-dimensional shape measurement with cone beam projection. In this method, changes in the shape of the optical sectioning profiles are quantified and fitted while scanning a calibration board in the depth direction, using a cubic function. In accuracy tests using a flat plate and a ring reference gauge, the proposed method obtains an accuracy of 0.02 mm in the depth dimension and 0.09 mm in the radial dimension. These results represent 88% and 55% improvements compared to previous analysis. For medical applications, an ear examination simulator was employed, and our measurement results were compared to ground truth data obtained by microfocus X-ray computed tomography. The surface deviation of our method relative to the ground truth data was ±0.36mm during manual operation. A comparison of the measurement results before and after calibration revealed an improvement in the peak agreement with the ground truth data, with the deviation shifting from 0.2 mm to −0.05mm. Our strategy achieves a digital transformation of 3D endoscopy, which would benefit a number of medical fields.

A phase error compensation method for three-dimensional shape measurement using improved iteration algorithm

The binary defocusing has been extensively studied in the three-dimensional measurement. But if the projector is slightly defocused, the binary fringes after defocusing still contain high-order harmonics compared with the ideal sinusoidal fringes, so the phase error caused by the nonlinear response is not negligible. In this paper, two models are proposed to calculate and compensate phase error, which include a double-precision iterative compensation model (DPICM) and a dual-domain iterative compensation model (DDICM). These two models obtain accurate phase errors by fusing the phases at different precision and domains. DPICM is composed of the double precision method and the improved iterative algorithm, and DDICM consists of the dual-domain method and the improved iterative algorithm. This improved iterative algorithm is used to compensate phase error, which can improve phase accuracy. DPICM and DDICM reduce the RMS error by 25.5% and 13.5% respectively.

Lightweight and edge-preserving speckle matching network for precise single-shot 3D shape measurement

Single-shot three-dimensional (3D) shape measurement method based on digital speckle correlation (DSC) has high measurement precision and only requires one pair of speckle images. Nevertheless, the matching of speckles is time-consuming and may lead to some data defects at large gradient variations and edge regions. This paper proposes an end-to-end speckle matching network to achieve fast, precise, and edge-preserved 3D measurement. Compared with other networks, the proposed network reduces the redundant channels in the Siamese feature-extraction subnetwork and constructs cost volume by group-wise correlation, making it possible to provide efficient representations at low computational consumption, thus being suitable for high-precision 3D measurement in large resolution. A lightweight 3D stacked hourglass subnetwork is used to aggregate cost information and furthermore, an edge refinement module is integrated by leveraging the left speckle image as a guide to regular edge details. A reliable speckle binocular dataset is prepared with a specialized device in a simple procedure and the labels are produced from the DSC method. Experiments demonstrate that the proposed network can produce dense disparity maps with subpixel precision and reduce single-shot inference time to 0.13 s with less GPU occupation, which is a significant improvement compared with the traditional DSC method and other learning-based methods. Moreover, the experiment validates that measurement precision of the reconstructed plane can reach less than 0.03 mm, meeting the requirements of most industrial applications.

Three-dimensional shape measurement method based on composite cyclic phase coding.

Phase coding is widely used in 3D measurement due to its good anti-interference and robustness. However, the measurement accuracy is affected by the limitation of the number of codewords. To solve this problem, we propose a 3D shape measurement method based on composite cyclic phase coding. The traditional phase coding is quantized cyclic without adding extra patterns, further adopting composite coding, using the composite cyclic phase coding grayscale values to distinguish the same cyclic codewords, and finally integrating them into a new fringe order sequentially for phase unwrapping to achieve effective expansion of codewords. The related experimental results show that the proposed method stably achieves high accuracy 3D reconstruction, which overcomes the misjudgment of codewords caused by traditional phase coding under high-frequency fringes due to system nonlinearity and noise. Meanwhile, compared with the improved phase coding method of temporal domain combined with spatial domain information, such as the method of quantized phase coding and connected region labeling, it can effectively avoid the phenomenon of error propagation, with high robustness and low algorithm complexity.

Calibration method for a structured light system based on DIC.

Structured light is a non-contact three-dimensional shape measurement method. The structured light system based on diffractive optical elements (DOEs) is widely used due to its low cost and compact structure. However, compared with a time sequence coded structured light system based on a digital projector, its projector cannot change the content, so it cannot be calibrated with a phase-shift-based method. This paper proposes a calibration method based on the pseudo-camera method and digital image correlation (DIC). It is suitable not only for the calibration of systems based on time series coding but also for the calibration of a speckle structured light system based on DOEs. Validation experiments were conducted in which our method achieved a reprojection error of 0.68 pixels in calibration and a radius error of less than 1% in a measurement of a cylinder in a 20cm×10cm field. Compared with existing calibration methods, this method does not need to set a datum plane, takes fewer photos, operates simply, and has higher calibration efficiency.

Digital image correlation assisted absolute phase unwrapping.

This paper presents an absolute phase unwrapping method for high-speed three-dimensional (3D) shape measurement. This method uses three phase-shifted patterns and one binary random pattern on a single-camera, single-projector structured light system. We calculate the wrapped phase from phase-shifted images and determine the coarse correspondence through the digital image correlation (DIC) between the captured binary random pattern of the object and the pre-captured binary random pattern of a flat surface. We then developed a computational framework to determine fringe order number pixel by pixel using the coarse correspondence information. Since only one additional pattern is used, the proposed method can be used for high-speed 3D shape measurement. Experimental results successfully demonstrated that the proposed method can achieve high-speed and high-quality measurement of complex scenes.

Dynamic Three-Dimensional Shape Measurement Method Based on Misaligned Gray Code

Dynamic Three-Dimensional Shape Measurement Method Based on Misaligned Gray Code

PMENet: phase map enhancement for Fourier transform profilometry using deep learning

Fringe projection profilometry (FPP) is a three-dimensional (3D) shape measurement method that involves projecting fringe patterns onto the object. A phase map retrieved from these fringe images is used for reconstructing the 3D surface of the object. Fourier transform and phase-shifting are two of the widely used fringe analysis techniques for performing 3D shape measurement using FPP. Fourier transform profilometry (FTP) has the advantage of performing high-speed measurement due to its single-shot nature. However, the reconstructed 3D surface has artifacts and high noise, specifically on the edges. On the other hand, phase-shifting profilometry (PSP) has the advantage of higher accuracy (relatively less noise level) but compromises on measurement speed. In this research, we propose a deep learning method to enhance the quality of the FTP phase maps using an efficient deep learning model called phase map enhancement net (PMENet). PMENet takes an FTP phase map as input and predicts a high-quality phase map in a supervised manner by using the phase maps obtained from 18-step PSP as ground truth. The training dataset was generated using a virtual FPP system. Validations were conducted on both the simulated data (generated by virtual FPP system) and the real-world data. The experimental results demonstrate that the trained neural network model can successfully improve the quality of the 3D geometric reconstruction with FTP and reduced the mean and root-mean-square error by 66% and 43%, respectively.

Three-dimensional shape measurement based on speckle-embedded fringe patterns and wrapped phase-to-height lookup table

A three-dimensional (3D) shape measurement method based on speckle-embedded fringe patterns (composite fringe patterns) and wrapped phase-to-height lookup table (LUT) is proposed. As an auxiliary signal, speckle is embedded into the phase domain of the projected fringes, and the wrapped phase and deformed speckle modulated by the objects are demodulated by phase-shifting algorithm. With the assistance of the LUT established in advance, the candidate heights are directly obtained by the wrapped phase. Subsequently, the speckle correlation is used to eliminate the height ambiguity, and then the absolute spatial height of the measured objects can be reconstructed without projecting any additional pattern. In this paper, the cooperation of embedded speckle and LUT allows the speckle to do correlation operation in some certain smaller area of the candidate heights’ location, instead of matching within the whole measurement range, thus it can improve the computational efficiency in the determination of the unique accurate height. Theoretical analysis and experimental results have demonstrated that the proposed method can achieve high-accuracy and robust 3D shape measurement of dynamic and static complex scenes.

Observation of micro-characters using three-dimensional shape measurement method based on speckle interferometry

Speckle interferometry has long been used to measure the deformation of an object with a rough surface. Currently, it can be used to measure the shape of a three-dimensional microstructure by intentionally applying a lateral shift to the measured object and analysing the change in phase due to the shift. Furthermore, the validity of this measurement method has been verified in terms of the measurement of microstructures, such as micro-silica spheres and diffraction gratings. In this study, the validity of the measurement of an object possessing an artificial shape by this method in the micro-region is analysed using micro-characters drawn by an electron-beam lithography system. It is confirmed that this method can perform the shape measurement of three-dimensional microstructures that are artificially produced. This was achieved by observing the letter ‘U’, illustrated with a line width beyond the diffraction limit of the objective lens.

Precise wide-range three-dimensional shape measurement method to measure superfine structures based on speckle interferometry

With regard to the three-dimensional (3D) shape measurement, based on speckle interferometry, of an object with a fine structure beyond the diffraction limit of the objective lens, phase distribution of each speckle region in a speckle pattern and a specklegram is discussed. The experimental results confirmed that the phase distribution with respect to the shape of an object exists spatially even in the speckle region of the speckle pattern. However, the measurement results of a superfine structure by speckle interferometry indicated that the spatial continuity of the phase distribution does not exist in each speckle but in the specklegram. This feature was used to confirm that a wide range of 3D shape measurements of an object with a fine structure can be realized by analyzing specklegrams calculated from speckle patterns before and after a lateral shift in speckle interferometry. Furthermore, it is confirmed that the shape measurement of a nonperiodical structure can be performed.

复合表面三维形貌测量方法的仿真与验证

复合表面三维形貌测量方法的仿真与验证

3D shape measurement based on structured light field imaging.

In this paper, a three-dimensional (3D) shape measurement method based on structured light field imaging is proposed, which contributes to the biomedical imaging. Generally, light field imaging is challenging to accomplish the 3D shape measurement accurately, as the slope estimation method based on radiance consistency is inaccurate. Taking into consideration the special modulation of structured light field, we utilize the phase information to substitute the phase consistency for the radiance consistency in epi-polar image (EPI) at first. Therefore, the 3D coordinates are derived after light field calibration, but the results are coarse due to slope estimation error and need to be corrected. Furthermore, the 3D coordinates refinement is performed based on relationship between the structured light field image and DMD image of the projector, which allows to improve the performance of the 3D shape measurement. The necessary light field camera calibration is described to generalize its application. Subsequently, the effectiveness of the proposed method is demonstrated with a sculpture and compared to the results of a conventional PMP system.

Enhanced phase-coding method for three-dimensional shape measurement with half-period codeword.

The phase-coding method has been widely used for 3D shape measurement, which uses sinusoidal phase-shifting patterns to recover the wrapped phase and the stair phase-coding patterns to determine the fringe order. However, due to random noises and image blurring, the fringe order is always misaligned with the wrapped phase, which will lead to fringe order errors. This paper presents an enhanced phase-coding method to address this misalignment problem by using half-period codewords, in which each codeword is aligned to the half-period of the sinusoidal patterns. Then, two complementary fringe orders with half-period dislocation can be calculated, which can effectively eliminate the fringe order errors. To extend the coding range of stair phase, this paper further develops a computational scheme based on the geometric constraint method. Simulations and experiments have been carried out, and their results confirm that the enhanced method can reliably recover the 3D shape of the measured objects.

Real-time high-dynamic-range fringe acquisition for 3D shape measurement with a RGB camera

This paper introduces a novel real-time, high-dynamic-range (HDR) three-dimensional (3D) shape measurement method using a RGB camera. The proposed method takes advantage of two effects to realize HDR 3D fringe acquisition for 3D shape measurements: (1) the different responses of a RGB camera’s three channels to a single color light source can be utilized to produce fringe images with different intensity levels; (2) the projector’s dark time can be utilized to produce bright versus dark intensity contrast if bit-wise binary patterns are used. Experiments demonstrate the real-time capabilities of the proposed method by dynamic 3D shape measurements (with a maximum camera frame rate of 166 Hz). Given that a bit-wise defocused binary pattern projection is adopted, the proposed HDR method has the potential for high-speed applications if a high-speed color camera is present.

Three-dimensional shape measurement based on binary fringe conventional projection

A three-dimensional (3D) shape measurement method based on binary fringe conventional projection is proposed. In traditional 3D shape measurement methods based on binary fringe projection, the binary fringe is used as an approximate sinusoidal fringe by defocusing projection or even directly used as an approximate sinusoidal fringe by nonlinear error suppression feature of large-step phase measuring profilometry (PMP). In the proposed method (Prop), neither defocusing projection is needed nor large-step PMP must be used. When the binary fringe is conventionally projected on the measured object, just by filtering operation in spatial frequency domain from the captured deformed pattern, the nearly unbroken sinusoidal deformed pattern can be extracted efficiently. While N ( N≥3) frames binary fringes with 1/ N period misalignment between each adjacent fringe are conventionally projected onto the measured object sequentially, the corresponding nearly unbroken N-step sinusoidal deformed patterns extracted from the N frames captured deformed patterns can be used to reconstruct the 3D shape of the measured object with N-step PMP algorithm. The experimental results show the feasibility and validity of the Prop. It is more flexible for its optional fringe period and optional phase-shifting steps. It has potential application in the real-time 3D shape measurement.

A fast reconstruction method for three-dimensional shape measurement using dual-frequency grating projection and phase-to-height lookup table

Phase-to-height mapping is indispensable part of three-dimensional (3D) shape measurement system based on phase analysis, which guarantees the accuracy of 3D reconstruction. In this paper, a fast 3D shape reconstruction method based on dual-frequency grating projection and phase-to-height lookup table is proposed. In this proposed method, the reference plane is moved with known interval along the measurement depth direction to establish the mapping lookup table between the phase of the high and low frequency projected grating and their corresponding spatial height pixel by pixel respectively. Then, two methods to establish the lookup table with the unwrapped and the wrapped phase have been respectively introduced according to the characteristics of dual-frequency grating. The actual experimental results show that the two methods can effectively extend the measurement range of the current wrapped phase-to-height lookup method, and the reconstruction accuracy is better than the traditional phase-to-height quadratic fitting method. At the same time, the consumed time for 3D shape reconstruction of the tested object by the two methods is compared and analyzed. Finally, combining the high performance parallel computation of GPU, the fast 3D shape reconstruction is realized with the speed of 60fps.

Three-dimensional shape measurement beyond the diffraction limit of lens using speckle interferometry

ABSTRACTSpeckle interferometry is generally known as a method for measuring the deformation of an object with rough surfaces. In this paper, a three-dimensional (3D) shape measurement method is proposed for superfine structures beyond the diffraction limit using the basic property of speckle interferometry. Since the differential coefficient distribution of the shape of such an object can be detected in speckle interferometry by imparting a known lateral shift to the measured object, the shape can be reconstructed by integrating the differential coefficient distribution. Based on experimental results obtained using diffraction gratings as measured objects, it is confirmed that the proposed method can measure 3D shapes that are beyond the diffraction limit of the lens.

Three-dimensional shape measurement method based on sinusoidal and triangular fringe projection

Three-dimensional shape measurement method based on sinusoidal and triangular fringe projection

Cross-sectional particle measurement in the resonance domain on the substrate through scatterometry.

We developed a versatile method for three-dimensional shape measurement where a specific particle can be selected on the substrate and its cross-sectional shape and size can be measured. A non-contact fast measurement is possible for the particle in the resonance domain. We applied rigorous coupled-wave analysis to the particle and calculated the diffraction patterns, comparing the patterns with the experimental results to obtain the size and shape. The shape and position of the focusing spot on the scattering particle was controlled precisely. With this method, the category of the analyzable object is extended to more shapes, such as rectangles and triangles, in addition to a conventional ellipsoid.