Structured Light Technique Research Articles

Three-Dimensional Registration for Handheld Profiling Systems Based on Multiple Shot Structured Light.

In this article, a multi-view registration approach for the 3D handheld profiling system based on the multiple shot structured light technique is proposed. The multi-view registration approach is categorized into coarse registration and point cloud refinement using the iterative closest point (ICP) algorithm. Coarse registration of multiple point clouds was performed using relative orientation and translation parameters estimated via homography-based visual navigation. The proposed system was evaluated using an artificial human skull and a paper box object. For the quantitative evaluation of the accuracy of a single 3D scan, a paper box was reconstructed, and the mean errors in its height and breadth were found to be 9.4 μm and 23 μm, respectively. A comprehensive quantitative evaluation and comparison of proposed algorithm was performed with other variants of ICP. The root mean square error for the ICP algorithm to register a pair of point clouds of the skull object was also found to be less than 1 mm.

Calibration of radial lens distortion using a Gaussian model

We present a lens distortion model based on the Gaussian function. The model is a potential source function of the popular Brown distortion model and requires the practical estimation of one Boolean and one real parameter. We also present a general technique for lens distortion identification, which consists of three steps: image data acquisition, localization and indexation/matching of image features, and the estimation of distortion parameters. The method uses a structured light technique, in which bar patterns are indexed by a Gray code. Acquired images are automatically processed using a multistep approach that localizes and indexes calibration points. An iterative analysis of differences between localizations of distorted points and their undistorted counterparts is proposed to estimate distortion model parameters. To compute undistorted localizations, the method estimates a homography matrix that is based on both undistorted data and iterative processing of distorted coordinates, which improves compensation accuracy. Experiments with three cameras show that the indexing strategy significantly decreases compensation error (from 0.26 to 0.09 pixels). The newly introduced Gaussian model is shown to be slightly more accurate and considerably more stable than the popular Brown model.

A novel imaging-enhancement-based inspection method for transparent aesthetic defects in a polymeric polarizer

The transparent aesthetic defects of polymeric polarizers are difficult to image and characterize using conventional illumination. To inspect such special defects, an automated inspection method was studied that employs the structured lighting technique to markedly enhance defect imaging. The defect was modelled as a microscale plano-convex lens, which has a slight difference in refractive index within its normal region. The imaging enhancement mechanism of the defect was simulated with a model in which the binary stripe patterns of the structured-light source are equivalent to a one-dimensional diaphragm. The experimental results were in agreement with the simulation, suggesting that the model and mechanism are feasible. A novel spatial texture filtering algorithm with higher speed and higher accuracy is proposed to process the structured light images. Approximately 200 samples with defects were successfully imaged, processed and characterized. This inspection method was verified by the final experimental results and has potential for real-time and in situ testing of defects in other polymer films and products.

Refining Geometry from Depth Sensors using IR Shading Images

We propose a method to refine geometry of 3D meshes from a consumer level depth camera, e.g. Kinect, by exploiting shading cues captured from an infrared (IR) camera. A major benefit to using an IR camera instead of an RGB camera is that the IR images captured are narrow band images that filter out most undesired ambient light, which makes our system robust against natural indoor illumination. Moreover, for many natural objects with colorful textures in the visible spectrum, the subjects appear to have a uniform albedo in the IR spectrum. Based on our analyses on the IR projector light of the Kinect, we define a near light source IR shading model that describes the captured intensity as a function of surface normals, albedo, lighting direction, and distance between light source and surface points. To resolve the ambiguity in our model between the normals and distances, we utilize an initial 3D mesh from the Kinect fusion and multi-view information to reliably estimate surface details that were not captured and reconstructed by the Kinect fusion. Our approach directly operates on the mesh model for geometry refinement. We ran experiments on our algorithm for geometries captured by both the Kinect I and Kinect II, as the depth acquisition in Kinect I is based on a structured-light technique and that of the Kinect II is based on a time-of-flight technology. The effectiveness of our approach is demonstrated through several challenging real-world examples. We have also performed a user study to evaluate the quality of the mesh models before and after our refinements.

Bioengineering Applications of 3D Scanning and Reconstruction Using a Depth Sensor

Paper approaches some characteristics and bioengineering applications of a handheld depth sensor for low-cost 3D scanning and reconstruction. The Kinect depth sensor used in this work was launched on June 2009 and was based around a gaming webcam peripheral. The Kinect sensor uses a structured light technique in order to develop real-time 3D surfaces. The 3D model of anatomic surface may have a lot of bioengineering applications. Some observations and comparisons are presented in connection with the scanning and 3D reconstruction of different anatomic surfaces.

Estimation of the Elastic Properties of Polymer Plates Using a Structured Light Technique

An experimental device for determining Young’s modulus of polymer plates is presented. Two sets of polymer blends were investigated. First samples were made of ethylene propylene diene (EPDM), high density polyethylene (HDPE), and meleic anhydride grafting on polyethylene (PE-MA) reinforced with glass fiber. The second set of samples was made of low density polyethylene (LDPE) reinforced with nylon fiber. Young’s modulus of a circular polymeric plate under biaxial stress was determined by measuring its out-of-plane displacement. This test is also known as bulge test. Values of Young’s modulus of the polymer plates were compared to those obtained from standard uniaxial tensile tests. Biaxial stress causes tension in all internal reinforcing fibers of the samples, in contrast tensile tests that cause tension mainly in fibers aligned with the applied force. Out-of-plane displacement was measured applying a laser triangulation setup based on a projected laser line and a CCD camera. A mathematical model for plates under the bulge test was then used in Young’s modulus calculations using out-of-plane displacement and plate dimensions. It is proved that the bulge test is more sensitive to reinforcement fiber compared to the standard uniaxial tensile test.

Accurate Dynamic 3D Sensing With Fourier-Assisted Phase Shifting

Phase shifting profilometry (PSP) and Fourier transform profilometry (FTP) are two well-known fringe analysis methods for 3D sensing. PSP offers high accuracy but requires multiple images; FTP uses a single image but is limited in its accuracy. In this paper, we propose a novel Fourier-assisted phase shifting (FAPS) method for accurate dynamic 3D sensing. Our key observation is that the motion vulnerability of multi-shot PSP can be overcome through single-shot FTP, while the high accuracy of PSP is preserved. Moreover, to solve the phase ambiguity of complex scenes without additional images, we propose an efficient parallel spatial unwrapping strategy that embeds a sparse set of markers in the fringe patterns. Our dynamic 3D sensing system based on the above principles demonstrates superior performance over previous structured light techniques, including PSP, FTP, and Kinect.

Construction and Optical Testing of Inflatable Membrane Mirror Using Structured Light Technique

Construction and characterization of an inflatable mirror prototype made out of flexible polymeric membranes are being presented. Surfaces were curved by imposing a slight excess of air pressure. Lightweighted, lowcost, and commercially available materials were selected in order to produce solar concentration elements at competitive prices. In this sense, large-area, image-forming mirrors with low optical acuity were achieved by concentration purposes. Optical characterization of the mirror’s shape at a given pressure or curvature radius was done by means of a structuredlight technique with a resolution of 0.1 mm finding a conical shape acquired by the inflated mirror as the best approximation. Concentration ratio achieved for a focal length of 5068 mm was of 25.1 suns, making a promising approach for lowering initial investment costs in applications such as hot-water, parabolic dish with Stirling engines, or concentrated photovoltaic electricity generation.

A robust method to extract a laser stripe centre based on grey level moment

Abstract Extraction of a laser stripe centre from an image is one of the most important measurement procedures when using structured light techniques. However, due to the influence of noise, extracting a light stripe centre accurately and stably remains a challenging problem. In this paper, a robust laser stripe centre extraction method is proposed. The method involves two steps: First, a closed-form solution for extracting the centre from every transversal section of a laser stripe is derived based on conservation laws of grey level moment. Second, a smoothing spline algorithm is used to determine a continuous curve of centre points. This algorithm can eliminate noise and maintain the original features of the discrete points. In the experiment, the proposed method is compared with recently reported techniques by detecting the same image. The results demonstrate that the method is more accurate and robust in the presence of noise.

Using physically Based Rendering to Benchmark Structured Light Scanners

Structured light scanning is ubiquituous in 3D acquisition. It is capable of capturing high geometric detail at a low cost under a variety of challenging scene conditions. Recent methods have demonstrated robustness in the presence of artifacts due to global illumination, such as inter-reflections and sub-surface scattering, as well as imperfections caused by projector defocus. For comparing approaches, however, the quantitative evaluation of structured lighting schemes is hindered by the challenges in obtaining ground truth data, resulting in a poor understanding for these methods across a wide range of shapes, materials, and lighting configurations. In this paper, we present a benchmark to study the performance of structured lighting algorithms in the presence of errors caused due to the above properties of the scene. In order to do this, we construct a synthetic structured lighting scanner that uses advanced physically based rendering techniques to simulate the point cloud acquisition process. We show that, under conditions similar to that of a real scanner, our synthetic scanner replicates the same artifacts found in the output of a real scanner. Using this synthetic scanner, we perform a quantitative evaluation of four different structured lighting techniques - gray-code patterns, micro-phase shifting, ensemble codes, and unstructured light scanning. The evaluation, performed on a variety of scenes, demonstrate that no one method is capable of adequately handling all sources of error - each method is appropriate for addressing distinct sources of error.

Relationship between fermentation behavior, measured with a 3D vision Structured Light technique, and the internal structure of bread

The bread-making process is a set of operations where could be relevant to use monitoring methods. Specifically, the fermentation phase is a crucial step in which the quality of the product can be affected. Several methods have been developed to monitor this stage based on different technologies. The aim of this study was to analyze and obtain information about the internal structure of bread dough during the fermentation process using a 3D vision system based on Structured Light (SL). The differences about fermentation behavior of two wheat flours classified as “high strength flour” by company providing were studied. The parameters of the internal structure of the baked product (final bubble size and their population density) were analyzed with 2D image segmentation. An important correlation (r=0.865) was observed between the 3D and 2D information, specifically between the transversal area and height (3D), and final bubble size and number of bubbles (2D). Although at the end of the dough fermentation process (Tf) the area (A) and maximum height (H) were different, the relationship between both parameters were similar, reaching a similar bubble size as a consequence of coalescence phenomena, independent of the bubble growth rate. This could be a base for the development of prediction models and devices to monitor the fermentation phase of the bread-making process.

Building Three-Dimensional Scanner Based on Structured Light Technique Using Fringe Projection Pattern

This paper presents the design and construction of a low-cost active noncontact three-dimensional (3D) scanner using the structured light method. The applied method is a generalization of the triangulation technique using a fringe projection pattern (by Gray code technique) to achieve a higher operating speed. The hardware equipment includes a single digital camera and a video projector. matlab was used for the calibration operation, the production of images by the above method, the image processing, and data calculation; while the Geomagic Qualify software was used to estimate the accuracy of the three-dimensional model obtained after scanning and to compare it with the tested object. Regarding the point cloud of the test object, the results demonstrate the precision of the used method and the possibility of developing a 3D scanner with a dimensional accuracy of ±0.05 mm.

Continuous monitoring of bread dough fermentation using a 3D vision Structured Light technique

Fermentation of the dough is an important phase in the bread-making process which is affected by several important factors related to raw materials and processing. Changes in fermentation affect parameters in the final product, such as texture, palatability and general quality. For this reason, it is important to develop dynamic methods to study this phase. In this work, a 3D vision system based on Structured Light (SL) was used to monitor the fermentation phase. The evolution of the dough was studied employing 10 wheat flours with non-physicochemical and rheological differences. However, differences in dough behaviors during fermentation were found based on SL method parameters. When the variation of the total transversal area was related to the maximum height at each fermentation time a set of peaks and valleys appeared. These sets were directly related to the fermentation capacity. Specifically, a lower number of peaks during the main fermentation time (100min) is related to wheat flours with high fermentation capacity. Consequently, the proposed SL Technique could be used as a method to check the fermentation capacity of wheat flours according to their fermentation behavior.

High-resolution, High-speed, Three-dimensional Video Imaging with Digital Fringe Projection Techniques

Digital fringe projection (DFP) techniques provide dense 3D measurements of dynamically changing surfaces. Like the human eyes and brain, DFP uses triangulation between matching points in two views of the same scene at different angles to compute depth. However, unlike a stereo-based method, DFP uses a digital video projector to replace one of the cameras1. The projector rapidly projects a known sinusoidal pattern onto the subject, and the surface of the subject distorts these patterns in the camera’s field of view. Three distorted patterns (fringe images) from the camera can be used to compute the depth using triangulation.Unlike other 3D measurement methods, DFP techniques lead to systems that tend to be faster, lower in equipment cost, more flexible, and easier to develop. DFP systems can also achieve the same measurement resolution as the camera. For this reason, DFP and other digital structured light techniques have recently been the focus of intense research (as summarized in1-5). Taking advantage of DFP, the graphics processing unit, and optimized algorithms, we have developed a system capable of 30 Hz 3D video data acquisition, reconstruction, and display for over 300,000 measurement points per frame6,7. Binary defocusing DFP methods can achieve even greater speeds8.Diverse applications can benefit from DFP techniques. Our collaborators have used our systems for facial function analysis9, facial animation10, cardiac mechanics studies11, and fluid surface measurements, but many other potential applications exist. This video will teach the fundamentals of DFP techniques and illustrate the design and operation of a binary defocusing DFP system.

High-resolution, High-speed, Three-dimensional Video Imaging with Digital Fringe Projection Techniques

Digital fringe projection (DFP) techniques provide dense 3D measurements of dynamically changing surfaces. Like the human eyes and brain, DFP uses triangulation between matching points in two views of the same scene at different angles to compute depth. However, unlike a stereo-based method, DFP uses a digital video projector to replace one of the cameras1. The projector rapidly projects a known sinusoidal pattern onto the subject, and the surface of the subject distorts these patterns in the camera’s field of view. Three distorted patterns (fringe images) from the camera can be used to compute the depth using triangulation. Unlike other 3D measurement methods, DFP techniques lead to systems that tend to be faster, lower in equipment cost, more flexible, and easier to develop. DFP systems can also achieve the same measurement resolution as the camera. For this reason, DFP and other digital structured light techniques have recently been the focus of intense research (as summarized in1-5). Taking advantage of DFP, the graphics processing unit, and optimized algorithms, we have developed a system capable of 30 Hz 3D video data acquisition, reconstruction, and display for over 300,000 measurement points per frame6,7. Binary defocusing DFP methods can achieve even greater speeds8. Diverse applications can benefit from DFP techniques. Our collaborators have used our systems for facial function analysis9, facial animation10, cardiac mechanics studies11, and fluid surface measurements, but many other potential applications exist. This video will teach the fundamentals of DFP techniques and illustrate the design and operation of a binary defocusing DFP system.

A Coded Structured Light System Based on Primary Color Stripe Projection and Monochrome Imaging

Coded Structured Light techniques represent one of the most attractive research areas within the field of optical metrology. The coding procedures are typically based on projecting either a single pattern or a temporal sequence of patterns to provide 3D surface data. In this context, multi-slit or stripe colored patterns may be used with the aim of reducing the number of projected images. However, color imaging sensors require the use of calibration procedures to address crosstalk effects between different channels and to reduce the chromatic aberrations. In this paper, a Coded Structured Light system has been developed by integrating a color stripe projector and a monochrome camera. A discrete coding method, which combines spatial and temporal information, is generated by sequentially projecting and acquiring a small set of fringe patterns. The method allows the concurrent measurement of geometrical and chromatic data by exploiting the benefits of using a monochrome camera. The proposed methodology has been validated by measuring nominal primitive geometries and free-form shapes. The experimental results have been compared with those obtained by using a time-multiplexing gray code strategy.

Accuracy enhancement of three-dimensional reconstruction using phase-shifting shadow moiré

An iterative algorithm and a denoising method for accuracy enhancement of three-dimensional (3-D) reconstruction processes by shadow moire is proposed. Using traditional shadow moire for 3-D reconstruction leads to a nonuniform phase shift error. The proposed iterative algorithm eliminates this error by using three phase-shifted interferograms and obtains precise phase information. Moreover the proposed denoising method, which is based on undecimated wavelet transform, effectively eliminates noise and grating patterns while retaining useful information. The proposed phase shifting shadow moire method is compared with the structured lighting method, which is a common method for 3-D reconstruction, and it is further compared with traditional shadow moire. The methods are simulated in a 3ds-Max environment. The simulation results show that the proposed shadow moire technique achieves greater accuracy in comparison with the traditional shadow moire and structured lighting techniques and also it has higher accuracy than existing typical phase-shifting algorithms.

Comparison of TOF and SL techniques for in-line measurement of food item volume using animal and vegetable tissues

A comparison between the techniques Time of flight (TOF) and Structured Light (SL), for in-line process, was carried out to determine the volume of several food types with different compositions, structures and dimensions; 2 meat tissue (lean meat and pork fat) and 3 vegetable tissue (potato, apple and avocado). The volume obtained was compared with that determined by physical measurements, employing two statistical methods (a Bland–Altman study and partial least square analysis). Results showed that Structured Light (SL) was a better technique than Time of flight (TOF) for determining volume. The TOF technique was affected by factors which were more influential when the S/V ratio (surface of the sample exposed to the camera and sample volume) increased. SL was slightly affected by the composition of the sample. Fat content and the level of unsaturated fatty acids could be the reason for the reflection of the laser light on the surface of the samples thereby reducing the accuracy. Even so, the values of R2 and RMSE of cross-validation, for the worst fit, demonstrated the quality of the SL technique.

초소형 카메라-프로젝터의 광학왜곡 보정을 이용한 위상변이 방식 3차원 스캐닝의 성능 향상

A distortion correction technique is presented to enhance the 3D scanning performance of a micro-size camera-projector system. Recently, several types of micro-size digital projectors and cameras are available. However, there have been few effort to develop a micro-size 3D scanning system. We develop a micro-sized 3D scanning system which is based on the structured light technique. Three images of phase-shifted sinusoidal patterns are projected, captured, and analyzed by the system to reconstruct 3D shapes of very small objects. To overcome inherent optical imperfection of the micro 3D sensor, we correct the vignetting and blooming effects which cause distortions in the phase image. Error analysis and 3D scanning results on small real objects are presented to show the performance of the developed 3D scanning system.

Small Obstacle Avoidance Sensor

This paper describes a laser ranging sensor that is suitable for applications like small unmanned aerial vehicles. The hardware consists of a diode emitter array and line-scan charge coupled devices. A structured-light technique measures ranges up to 30 meters for 64 field angles in a 90 degree field of view. Operation is eye safe, and the laser wavelength is not visible to night vision goggles. This paper describes a specific sensor design in order to illustrate performance for a given package size.