3D Surface Reconstruction Research Articles

Design considerations for rotation scanning projection moiré topography

Projection moiré topography is an optical technique for 3D surface reconstruction based on the geometric interference between two optical grids. To expand the measurement area and to maintain a high measurement resolution, a rotation scanning projection moiré profilometry technique is developed here. We present the mathematical description of an arbitrarily arranged projection moiré model, and derive the expression for phase and coordinates. On this basis, we establish a relation between the system parameters and rotation parameters, making it possible to calculate the system parameters and perform surface reconstruction for any rotation angle, using the same projection moiré model.

Mapping erosion and deposition in an agricultural landscape: Optimization of UAV image acquisition schemes for SfM-MVS

Structure-from-motion (SfM) and multi-view stereo (MVS) algorithms coupled with the use of unmanned aerial vehicles (UAVs) have become a popular tool in the geosciences for modelling complex landscapes on-demand allowing for high-resolution topographic change-detection studies to be conducted at minimal cost. To identify the effects of UAV image orientation on the accuracy of SfM-MVS 3D surface models, we tested four different UAV image acquisition schemes that incorporated both nadir and oblique imagery of an agricultural field. The coupling of nadir and oblique imaging angles led to the highest surface model accuracy in the absence of ground control points (GCPs; vertical RMSE: 0.047 m, horizontal RMSE: 0.019 m), while with a normative distribution of GCPs the nadir-only image sets had similar accuracy metrics (vertical RMSE 0.028 m, horizontal RMSE 0.017 m) to surface models generated with nadir and oblique imaging angles (vertical RMSE 0.028 m, horizontal RMSE 0.013 m). Homologous keypoint matching between nadir and oblique imagery was poor when the survey conditions were bright and the surface texture of the field was homogeneous, leading to broad-scale vertical noise in the generated surface models. Results indicate that a nadir-only image set accompanied with a dense deployment of GCPs is the most ideal for SfM-MVS agricultural 3D surface reconstructions. The diachronic analysis of surface models generated from nadir-only image sets were able to detect surface-change >0.040 m in depth (i.e., rill and gully erosion, depositional zones) and were comparable to results obtained from a terrestrial laser scanner. Stable GCPs should be used where possible to ensure precise co-registration between subsequent UAV surveys.

A Real-Time 3D Laparoscopic Imaging System: Design, Method, and Validation.

This paper aims to propose a 3D laparoscopic imaging system that can realize dense 3D reconstruction in real time. Based on the active stereo technique which yields high-density, accurate and robust 3D reconstruction by combining structured light and stereo vision, we design a laparoscopic system consisting of two image feedback channels and one pattern projection channel. Remote high-speed image acquisition and pattern generation lay the foundation for the real-time dense 3D surface reconstruction and enable the miniaturization of the laparoscopic probe. To enhance the reconstruction efficiency and accuracy, we propose a novel active stereo method by which the dense 3D point cloud is obtained using only five patterns, while most existing multiple-shot structured light techniques require [Formula: see text] patterns. In our method, dual-frequency phase-shifting fringes are utilized to uniquely encode the pixels of the measured targets, and a dual-codeword matching scheme is developed to simplify the matching procedure and achieve high-precision reconstruction. Compared with the existing structured light techniques, the proposed method shows better real-time efficiency and accuracy in both quantitative and qualitative ways. Ex-vivo experiments demonstrate the robustness of the proposed method to different biological organs and the effectiveness to lesions and deformations of the organs. Feasibility of the proposed system for real-time dense 3D reconstruction is verified in dynamic experiments. According to the experimental results, the system acquires 3D point clouds with a speed of 12 frames per second. Each frame contains more than 40,000 points, and the average errors tested on standard objects are less than 0.2mm. This paper provides a new real-time dense 3D reconstruction method for 3D laparoscopic imaging. The established prototype system has shown good performance in reconstructing surface of biological tissues.

Hybrid Framework for Haptic Texture Modeling and Rendering

We present a hybrid framework for the modeling and rendering of both homogeneous and inhomogeneous textures with high realism by combining force and vibration feedback. Given a real texture sample, we first capture a force-feedback model using photometric stereo, which is one of the most accurate algorithms for 3D surface reconstruction in computer vision. This method allows us to model the micro-geometry of the surface and represent it by a two-dimensional (2D) height map in a very high resolution to the order of 10$\mu \text{m}$ . We also provide a new force rendering algorithm adequate for such an accurate and fine texture model. By a user study, we verify that the proposed force modeling and rendering algorithms are suitable for our hybrid texture framework. Second, we model the homogeneous and stationary vibrational characteristics of the real texture sample using the prevalent LPC (linear predictive coding)-based method. Third, we merge the force and vibration feedback models appropriately into a hybrid framework on the basis of their spectral characteristics. Last, another user study that assessed the perceived similarity between real textures and their virtual simulations demonstrates that our hybrid framework can achieve a high level of realism. To our knowledge, our framework is unique in that it can model inhomogeneous (in additional to homogeneous) real textures and render them in a virtual environment with high realism by providing both force and vibrational sensory cues.

Reconstruction and registration of 3D surfaces obtained by scanning

In this article the general approach for the three-dimensional surface reconstruction is examined. Also, the registration of three-dimensional surfaces is considered. The process of converting a three-dimensional model from a triangular mesh into a point cloud based on the generation of a random point using the principle of barycentric coordinates was applied. Based on the material studied, a complex procedure is proposed, which includes both correction of errors and defects in surfaces obtained as a result of an object scanning, and the registration of these surfaces previously converted into point clouds. On a specific example of the usage of three-dimensional models in dentistry the proposed procedure was tested, results were obtained and conclusions were made. Figs.: 6. Refs.: 23 titles. Keywords: 3D-surface; reconstruction; registration; algorithm; scanning; triangular mesh; point cloud.

3D acquisition and stereo-camera calibration by active devices: A unique structured light encoding framework

The use of light encoding techniques is widespread in the field of 3D surface reconstruction. This paper presents a stereo-camera calibration methodology, which integrates structured light encoding with an active digital device. The structured light encoding approach is proposed to unambiguously solve the stereo matching issue for stereo-camera setups. A sequence of vertical and horizontal binary striped patterns, combined with a checkerboard pattern, is displayed by a high-resolution LCD screen, which is used as calibration board. A bundle adjustment technique is adopted to simultaneously adjust both camera parameters and screen geometry, as part of the stereo-camera calibration process, thus taking into account the possible inaccuracies of the digital display. The same structured light approach, with small variants, is projected by a multimedia digital projector to carry out 3D surface reconstruction. The proposed methodology defines a comprehensive framework for the development of a 3D optical scanner, from calibration to 3D acquisition, which has been validated by measuring primitive surfaces and reconstructing free-form shapes with different stereo-camera setups.

Reconstructing 3D point clouds in real time with look-up tables for structured light scanning along both horizontal and vertical directions.

By scanning static, not moving, objects along both the horizontal and vertical axes instead of one, structured light illumination achieves more accurate and robust 3D surface reconstructions but with greater latency on computing 3D point clouds. If scanning is performed along only one axis, it has been reported that look-up tables, manually derived from the calibration matrices of a camera and a projector, can significantly help to speed up computation; however, it has been nearly impossible to manually derive similar look-up tables for phases scanned along two axes. In this Letter, we bridge this divide by introducing the constraint of epipolar geometry to automatically compute look-up tables and thus, significantly speed up computing 3D point clouds with only basic arithmetic operations rather than time-consuming matrix computations. Experimental results show that the proposed method, using only single-thread CPU computing, reduces process latency by an order of magnitude.

Accurate and efficient 3D reconstruction system for the human body with color texture based on DIC

Reconstructing 3D human body models for real people benefits a variety of applications. However, it is challenging to reconstruct high-fidelity human body models with high-accuracy. This study presents an accurate and efficient 3D human body reconstruction system based on digital image correlation (DIC), which can simultaneously capture high-quality 3D human body model and high-fidelity 2D color texture. The 3D reconstruction system is set up with twenty-seven cameras and nine speckle-pattern projectors, which is able to acquire 3D shape data and 2D color data. Specifically, the nine binocular stereo vision systems are adopted to reconstruct 3D shape, where the 3D reconstruction is carried out based on stereo-DIC with a speckle-pattern projector. The other nine color cameras are utilized to capture 2D color texture. It only needs a single-shot to record the images required for 3D reconstruction, which takes about 1.25 ms to ensure the high-accuracy and high-efficiency of the proposed 3D reconstruction system. A high-precision multi-camera calibration method is employed to calibrate the proposed multi-camera system. The mapping relationship between the reconstructed 3D object point and the 2D color image is determined by the calibrated result, which makes the color texture acquisition precisely and effectively. The proposed system can achieve accurate and high-quality 3D reconstruction of the whole human body surface with high-fidelity color texture. Experimental results demonstrate the high-performance of the proposed 3D reconstruction system.

Preliminary Study on Development of 3D Free-Form Surface Reconstruction System Using a Webcam Imaging Technique

Reverse engineering faces challenges when it comes to data extraction using indirect contact method. There are 3D scanners capable of extracting data from distance more than a few meters away without contacting the objects but yet limited to same space range only. Although synchronizing the 3D scanners from another place can enable long distance data extraction, synchronizing devices over long distance can lead to other issues such as lagging. Moreover, the uses of 3D scanners usually involve quite long consumption time regardless of same space range or long distance range application. To overcome these issues, this study introduces a 3D surface reconstruction system by including a webcam imaging technique to produce a 3D point cloud, which is the same as that produced by a 3D scanner, and thus, eliminate the use of a 3D scanner. Through webcam imaging, image data can be transferred over the Internet and can be shared instantly even in different locations. Hence, webcam imaging is an alternative to using a 3D scanner. This system will continuously generate 3D free-form surfaces to resemble the surface profile of physical objects. Acquiring data until the surface fitting process will take less time compared with using a combination of 3D scanner and CAD software. In addition, results produced from the webcam imaging technique are compared with conventional results from a 3D scanner.

Development of a device for 3D surface reconstruction of thermal shrinkage of flame-resistant fabrics

Thermal shrinkage of flame-retardant fabrics under high temperature exposure would result in the change of air layer under clothing, which can affect the thermal protective performance of clothing. This paper introduces a new image-based device to characterize the thermal shrinkage of the fabrics. The device was aimed to reconstruct the three-dimensional (3D) surface morphology of fabrics based on grayscale images. The imaging collecting system was designed to obtain the grayscale images of thermal-shrinkage fabrics. Then, MATLAB programming of the 3D reconstruction algorithm based on photometric stereo was developed. A comparison of the results of the 3D reconstruction method with the 3D scanning method showed that the 3D reconstruction method was practical and repeatable. The research provides a simple and effective method to characterize the 3D morphology of thermal-shrinkage fabrics, which will help in the quantification of thermal shrinkage of fabrics and the prediction of the thermal protective performance of clothing.

Implicit surface reconstruction with radial basis functions via PDEs

We propose a partial differential equations-based algorithm for the 3D implicit surface reconstruction from a set of scattered cloud data. In the solution process, the method of approximate particular solutions with the IMQ radial basis function is employed for solving the modified Helmholtz or Poisson equation with a constant Dirichlet boundary condition. We also consider to repair the surfaces when a certain region of cloud data points are missing. The selection of several parameters is studied for the optimal recovery of the surfaces. Four examples are presented to validate the effectiveness of the proposed method.

Efficient Measurement of Key-Cap Flatness for Computer Keyboards With a Multi-Line Structured Light Imaging Approach

A computer keyboard is an important human–machine interaction device. A key-cap and a keyboard are flexibly connected, and its assembly quality, especially flatness, is crucial. In manufacturing quality control, the requirement for assembly quality of keyboards is quite strict, where the flatness measurement of key-caps is a key link. In this paper, an apparatus based on multi-line structured light imaging, and machine vision is designed for online rapid detection of key-cap flatness, is proposed. First, a novel 3D measurement scheme is presented to replace the complete 3D surface reconstruction of conventional methods. Then, an array of cameras with a perspective projection transformation matrix calibration approach is adopted to expand the scanning range of the 3D sensor for large-scale and high-resolution target imaging. In addition, a multi-step image processing is applied to accurately extract the centerline of the projected fringes on a key-cap with printed character, which indirectly helps to improve the measurement accuracy. Finally, a sparse sampling flatness estimation algorithm is proposed to improve the detection efficiency. According to experiments in real production lines, our apparatus can achieve a detection accuracy of 99.74% for various computer keyboards.

A Simple and Practical Single-Camera Stereo-Digital Image Correlation Using a Color Camera and X-Cube Prism.

A simple and practical full-frame single-camera stereo-digital image correlation (stereo-DIC) technique for three-dimensional (3D) shape, displacement, and deformation measurements is proposed. The technique uses a compact X-cube prism-based color separation device and a color camera to capture images of blue and red colors from different optical paths, and then extracts the surface 3D shape and deformation information of a test sample by processing the captured two sub-channel color images using regular stereo-DIC algorithm. Compared with the existing full-frame single-camera stereo-DICs, the proposed one eliminates the need for a beam splitter and two bandpass filters to capture images, and offers more simple, compact, and easy-to-use optical arrangement. This novel single-camera stereo-DIC technique was validated by a series of baseline experiments involving 3D surface reconstructions, translation tests, and full-field deformation measurements, which provide a new flexible and practical avenue for measuring surface 3D shape and deformation, particularly in microscopic and high-speed applications.

Epstein-Barr virus genome packaging factors accumulate in BMRF1-cores within viral replication compartments.

Productive replication of Epstein-Barr virus (EBV) during the lytic cycle occurs in discrete sites within nuclei, termed replication compartments. We previously proposed that replication compartments consist of two subnuclear domains: “ongoing replication foci” and “BMRF1-cores”. Viral genome replication takes place in ongoing replication foci, which are enriched with viral replication proteins, such as BALF5 and BALF2. Amplified DNA and BMRF1 protein accumulate in BMRF1-cores, which are surrounded by ongoing replication foci. We here determined the locations of procapsid and genome-packaging proteins of EBV via three-dimensional (3D) surface reconstruction and correlative fluorescence microscopy-electron microscopy (FM-EM). The results revealed that viral factors required for DNA packaging, such as BGLF1, BVRF1, and BFLF1 proteins, are located in the innermost subdomains of the BMRF1-cores. In contrast, capsid structural proteins, such as BBRF1, BORF1, BDLF1, and BVRF2, were found both outside and inside the BMRF1-cores. Based on these observations, we propose a model in which viral procapsids are assembled outside the BMRF1-cores and subsequently migrate therein, where viral DNA encapsidation occurs. To our knowledge, this is the first report describing capsid assembly sites in relation to EBV replication compartments.

Stereo-imaging LSPIV (SI-LSPIV) for 3D water surface reconstruction and discharge measurement in mountain river flows

Large-Scale Particle Image Velocimetry (LSPIV) has been successfully applied for stream flow and flood measurements. In most LSPIV studies, the water surface has been approximated as a two-dimensional planar surface. Errors may arise due to this approximation for flows in steep channels and with complex riverbed topography, especially for small mountain rivers. In this study, a Stereo Imaging based LSPIV system (SI-LSPIV) is developed to reconstruct the three-dimensional topography and water surface distribution, in addition to surface velocity measurements. The SI-LSPIV system consists of two synchronized Raspberry Pi camera modules that can capture stereo images autonomously. Uncalibrated rectification and point cloud methods are implemented for surface reconstruction. Detailed three-dimensional surface topography as well as water level distribution can be obtained, which facilitates image ortho-rectification and wetted-area delineation for LSPIV calculation. Over the past years, an unattended SI-LSPIV system has been deployed in Longxi River, Sichuan, China. One major flow event was captured and the results were presented to demonstrate that LSPIV analysis aided with three-dimensional water surface distribution can improve velocity measurement and stream discharge estimation for torrential mountain river flows.

3D surface topography imaging in SEM with improved backscattered electron detector: Arrangement and reconstruction algorithm

We propose a new SFS (shape from shading) technique for improved 3D surface reconstruction and imaging of relatively smooth surface topography using the scanning electron microscope (SEM). The new arrangement of backscattered electrons detector plates allows decreasing the initial energy of the electron probe, which makes this SEM technique to be suitable for usage on radiation-sensitive samples like biological tissues. Experiments show high effectiveness of the method, which improves both the gradient sensitivity of the signal and the signal to noise ratio.

An Improved RANSAC Surface Reconstruction Study

Surface reconstruction technology has always been an important research area in graphic image. However, efficiently processing massive amounts of high-precision point cloud data is still a problem worth studying. Based on the characteristics of point cloud data and the existing research results, this paper proposes a RANSAC algorithm based on voxel segmentation and applies it to 3D surface reconstruction. By analyzing the characteristics of the original point cloud data, the method improves the reconstruction speed of the 3D model while retaining a large amount of effective feature information and ensuring high precision. The experimental results show that compared with the reconstruction algorithm based on the original RANSAC algorithm, the proposed algorithm can effectively reduce the surface reconstruction time of 47.51%, and the model error after processing is only one thousandth compared with the original model.

Computed Tomography-Mediated Registration of Trapeziometacarpal Articular Cartilage Using Intraarticular Optical Coherence Tomography and Cryomicrotome Imaging: A Cadaver Study.

ObjectiveAccurate, high-resolution imaging of articular cartilage thickness is an important clinical challenge in patients with osteoarthritis, especially in small joints. In this study, computed tomography (CT) mediated catheter-based optical coherence tomography (OCT) was utilized to create a digital reconstruction of the articular surface of the trapeziometacarpal (TMC) joint and to assess cartilage thickness in comparison to cryomicrotome data.DesignUsing needle-based introduction of the OCT probe, the articular surface of the TMC joint of 5 cadaver wrists was scanned in different probe positions with matching CT scans to record the intraarticular probe trajectory. Subsequently and based on the acquired CT data, 3-dimensional realignment of the OCT data to the curved intraarticular trajectory was performed for all probe positions. The scanned TMC joints were processed using a cryomicrotome imaging system. Finally, cartilage thickness measurements between OCT and cryomicrotome data were compared.ResultsSuccessful visualization of TMC articular cartilage was performed using OCT. The CT-mediated registration yielded a digital reconstruction of the articular surface on which thickness measurements could be performed. A near-perfect agreement between OCT and cryomicrotome thickness measurements was found (r2 = 0.989).ConclusionThe proposed approach enables 3D reconstruction of the TMC articular surface with subsequent accurate cartilage thickness measurements, encouraging the development of intraarticular cartilage OCT for future (clinical) application.

HPM-TDP: An efficient hierarchical PatchMatch depth estimation approach using tree dynamic programming

Accurate and efficient estimation of the dense depth information from a pair of stereo images is a key step for many applications such as digital surface model production, 3D reconstruction and visualization, autonomous driving, and robotic navigation. Although great progress has been achieved in stereo matching over the past decade, the matching difficulties in poor and repetitive texture regions remain an issue. Aiming at solving the shortcomings of the current methods, this paper proposes HPM-TDP, which is an efficient hierarchical PatchMatch depth estimation approach that integrates a coarse-to-fine image pyramid strategy with a continuous Markov random field (MRF)-based global energy optimization framework, and minimizes the energy function by combining a hierarchical PatchMatch (HPM) framework and local α-expansion based tree dynamic programming (TDP). Firstly, the coarse-to-fine image pyramid strategy is integrated with the PatchMatch filter algorithm to quickly generate the hierarchical disparity plane prior for initializing each pixel’s disparity plane of the energy function optimization. Secondly, a multi-resolution cost aggregation strategy is adopted to boost the robustness of the matching cost function in the poor and repetitive texture areas. Finally, the HPM framework and local α-expansion based TDP are adopted to solve the non-submodular energy optimization problem, resulting in a globally optimized disparity plane map. Three benchmark datasets—the Middlebury 3.0, KITTI 2015, and Vaihingen datasets—were used to test the performance of HPM-TDP. The comprehensive experimental results demonstrate that HPM-TDP obtains a good performance on all datasets in terms of the (“Out-Noc”, “Avg-Noc”, “Out-All”, “Avg-All”) of (15.45%, 4.16px, 24.26%, 12.14px) and (5.46%, 1.20px, 6.55%, 1.54px) for Middlebury 3.0 and KITTI 2015 training datasets, and the (“Out-All”, “Avg-All”) of (26.32%, 4.04px) for Vaihingen dataset, respectively.

3D Reconstruction of Weld Pool Surface in Pulsed GMAW by Passive Biprism Stereo Vision

Geometrical information of the 3D weld pool surface provides clues to determine the penetration state whose monitoring and feedback control are crucial for critical applications involving high temperatures and pressures. In this letter, a biprism stereo vision system is established to sense the weld pool surface under different penetration states during pulsed gas metal arc welding (GMAW-P) with a V groove joint. Only one optical filter is used to block out the arc and capture a clear image of the weld pool during the base current period in GMAW-P. While most of the weld pool surface is textureless, there are a few features such as slag that are detectable. Hence, during the stereo matching sparse feature corners are detected and credibly matched in pairs, followed by a disparity region growing step to produce a semi-dense and unambiguous disparity map. Polynomial interpolation is, then, applied to obtain subpixel disparity for every matched pixel. Finally, the point cloud and triangle mesh representation of the weld pool surface with different penetration states, partial, full, and over penetration, are presented. The experimental results verified the effectiveness of the proposed biprism stereo vision method in monitoring the weld penetration state.