Structured Light Technology Research Articles

A Globally Consistent Merging Method for House Point Clouds Based on Artificially Enhanced Features

In the process of using structured light technology to obtain indoor point clouds, due to the limited field of view of the device, it is necessary to obtain multiple point clouds of different wall surfaces. Therefore, merging the point cloud is necessary to acquire a complete point cloud. However, due to issues such as the sparse geometric features of the wall point clouds and the high similarity of multiple point clouds, the merging effect of point clouds is poor. In this paper, we leverage artificially enhanced features to improve the accuracy of registration in this scenario. Firstly, we design feature markers and present their layout criteria. Then, the feature information of the marker is extracted based on the Color Signature of Histograms of OrienTations (Color-SHOT) descriptor, and coarse registration is realized through the second-order similarity measure matrix. After that, precise registration is achieved using the Iterative Closest Point (ICP) method based on markers and overlapping areas. Finally, the global error of the point cloud registration is optimized by loop error averaging. Our method enables the high-precision reconstruction of integrated home design scenes lacking significant features at a low cost. The accuracy and validity of the method were verified through comparative experiments.

Simulation method of asphalt mixture based on fracturable digital aggregate model

This study proposed an approach to construct a fracturable aggregate model to make the fracture patterns consistent with the real situation. Firstly, this study utilized three-dimensional structured light technology to obtain the geometric shell of aggregate and divide them into four categories of spherical, angular, elongated and flaky aggregates according to the minimum bounding box algorithm. Secondly, this study proposed a discrete-spheres filling algorithm based on the Bubble Pack Filling algorithm to fill the shell accurately. Thirdly, this study established the conversion equations between the macro and micro parameters of aggregate by using the linear parallel bonding model, and used it to establish the fracturable aggregate model. Through the indoor and virtual static fracture tests, this study found that the fracture patterns of the fracturable aggregate models are similar to the real situation and characterized the fracture patterns of four types of aggregates. Further analyzing the virtual static fracture tests, this study found that the bearing capacity of aggregates is determined by the compressive strength of host rocks and the shape of aggregate. Through the indoor and virtual uniaxial compression creep tests, this study investigated the behavior of the mixture specimens consisted by the fracturable aggregate model. The stress-strain curves of the fracturable and unfracturable aggregate model in this test were plotted in this study, and the results showed that the relative error between the results obtained by the fracturable model and those of the indoor test is much smaller than that of the unfracturable model. Considering that existing simulation tests on asphalt mixture often ignore aggregate deformation and fracture, so this study used the method mentioned above to obtain the fracturable aggregate model, which can improve the confidence of asphalt mixture simulation results.

Mobile phone applications for 3-dimensional scanning and digital anthropometry: a precision comparison with traditional scanners.

The precision of digital anthropometry through 3-dimensional (3D) scanning has been established for relatively large, expensive, non-portable systems. The comparative performance of modern mobile applications is unclear. Forty-six adults (age: 23.3 ± 5.3 y; BMI: 24.4 ± 4.1 kg/m2) were assessed in duplicate using: (1) a mobile phone application capturing two individual 2D images, (2) a mobile phone application capturing serial images collected during a subject's complete rotation, (3) a traditional scanner with a time of flight infrared sensor collecting visual data from a subject being rotated on a mechanical turntable, and (4) a commercial measuring booth with structured light technology using 20 infrared depth sensors positioned in the booth. The absolute and relative technical error of measurement (TEM) and intraclass correlation coefficient (ICC) for each method were established. Averaged across circumferences, the absolute TEM, relative TEM, and ICC were (1) 0.9 cm, 1.5%, and 0.975; (2) 0.5 cm, 0.9%, and 0.986; (3) 0.8 cm, 1.5%, and 0.974; and (4) 0.6 cm, 1.1%, and 0.985. For total body volume, these values were (1) 2.2 L, 3.0%, and 0.978; (2) 0.8 L, 1.1%, and 0.997; (3) 0.7 L, 0.9%, and 0.998; and (4) 0.8 L, 1.1%, and 0.996, with segmental volumes demonstrating higher relative errors. A 3D scanning mobile phone application involving full rotation of subjects in front of a smartphone camera exhibited similar reliability to larger, less portable, more expensive 3D scanners. In contrast, larger errors were observed for a mobile scanning application utilizing two 2D images, although the technical errors were acceptable for some applications.

Metasurface- and PCSEL-Based Structured Light for Monocular Depth Perception and Facial Recognition.

The novel depth-sensing system presented here revolutionizes structured light (SL) technology by employing metasurfaces and photonic crystal surface-emitting lasers (PCSELs) for efficient facial recognition in monocular depth-sensing. Unlike conventional dot projectors relying on diffractive optical elements (DOEs) and collimators, our system projects approximately 45,700 infrared dots from a compact 297-μm-dimention metasurface, drastically more spots (1.43 times) and smaller (233 times) than the DOE-based dot projector in an iPhone. With a measured field-of-view (FOV) of 158° and a 0.611° dot sampling angle, the system is lens-free and lightweight and boasts lower power consumption than vertical-cavity surface-emitting laser (VCSEL) arrays, resulting in a 5-10 times reduction in power. Utilizing a GaAs-based metasurface and a simplified optical architecture, this innovation not only addresses the drawbacks of traditional SL depth-sensing but also opens avenues for compact integration into wearable devices, offering remarkable advantages in size, power efficiency, and potential for widespread adoption.

Three-Dimensional Reconstruction Based on Multiple Views of Structured Light Projectors and Point Cloud Registration Noise Removal for Fusion.

Structured light technology is typical for capturing 3D point cloud data. This paper proposes a 3D reconstruction system to obtain point cloud data of complex objects based on nine-order Gray code and an eight-step structured light projection combined with a phase shift and phase unwrapping method. In this system, two projectors serve as bilateral projectors for structured light, along with a camera and rotating platforms. These components were used to obtain point cloud data from multiple perspectives, which helps avoid the shadow areas caused by a single projection angle and provides complementary point cloud data. The point clusters scanned under each perspective were transformed into the same coordinate system. Furthermore, a distance-based point cloud noise removal algorithm was proposed to optimize platform noise and facilitate point cloud data fusion. The experimental results proved that the system effectively captures 3D point cloud data for complex objects. The dimensional quantitative analysis of an aero engine blade was also performed.

3D scanning of foreign body causing injuries in children: an innovative approach to injury prevention

Abstract Background Foreign Body (FB) injuries in children are a relevant public health problem associated with significant morbidity and mortality. The characterization of objects causing the injuries is an essential step for developing effective prevention strategies. Scanning the objects causing the injuries would help identify characteristics that make the objects dangerous for young children. The study aims to use 3D scanning within a large series of FB injuries collected in the Susy Safe database to develop a system able to identify the risk profile of everyday objects based on information collected through the scanning study. Methods The Susy Safe is the largest international registry collecting data on FB injuries in children under the age of fifteen. It collects almost 35,000 cases. For each case, information about the child, the object, and the circumstances of the injuries are collected. Furthermore, whenever possible (according to the object type and the approach used for FB removal) also the object is collected. Such objects are undergoing 3D scanning with structured light technology, a common non-contact scanning method allowing for 3D digitization of all dimensions and appearances. Results Currently, 383 objects have undergone 3D scanning. Most objects were made of a stiff material (60%). The scanning of the objects is still ongoing. The digitalized information for each object is collected in a REDCap repository together with all information about the FB injury and the picture of the object. Conclusions The final aim of the project is to develop a web-based application able to identify dangerous objects, based on shape and size parameters, according to the data collected through the scanning. Its objective is to make people aware of the risks posed by particular objects, especially parents and guardians of children. Furthermore, through the web-based application, parents will be able to report injuries and share images on FBs. Key messages • 3D scanning would be a promising opportunity to improve primary prevention of foreign body injuries. • Integrating data from 3D scanning and surveillance systems would represent the basis for the development of a web application able to assist families in identifying hazardous objects.

Research on a high-precision extraction algorithm for aircraft skin seam features

The precision measurement of the surfaces of aircraft based on structured light technology is an important development direction of aircraft skin assembly and maintenance. Based on the characteristics of the point cloud in the skin seam area, the height difference mutation method and the slope value maximization method are proposed to extract the edge points of the feature point cloud, achieving the high-precision extraction of the flush and gap of the aircraft skin. First, the 3D point cloud data of the feature region are generated with fringe projection profilometry, and an adaptive matching calibration algorithm is used to calibrate the feature region with high precision. A 3D feature data calculation model is established for both straight line and curve seams, and the edge points of the feature point cloud are extracted with the height difference mutation method and the slope value maximization method to achieve the high-precision extraction of 3D feature data of aircraft skin seams. In this study, an aircraft seam 3D data measurement model is systematically established. This model is suitable for both straight lines and curves. Compared with the algorithm for the feature seam extraction of the slope difference skin, the model has better robustness, and the average measurement error is reduced to approximately one-third of that of the classical algorithm.

Characterization of the Nozzle Ablation Rate Based on 3D Laser Scanning System

The nozzle of solid rocket motor (SRM) is easily ablated by high temperature, high pressure, high speed, and corrosive particles, which affects the stability of rocket flight. Therefore, the measurement and characterization of the nozzle ablation rate are helpful in providing some guidance for the design of nozzle material and structure. However, due to the high surface roughness of the composite nozzle after ablation, it is difficult to obtain an accurate ablation rate by contact measurement methods. The 3D laser scanning system is a 3D non-contact measurement technology using structured light technology, phase measurement technology, and computer vision technology. It has the advantages of non-contact, large scanning size, flexibility, and portability. In this paper, a 3D reconstruction of the ablation nozzle is carried out based on the 3D laser scanning system. Additionally, the ablation rate of the nozzle is measured without cutting the actual specimen. Furthermore, the pressure, temperature, and surface convective heat transfer coefficient trends are numerically calculated and compared with the ablation rate. Additionally, the empirical formula between ablation rate and pressure, temperature, convective heat transfer coefficient is obtained empirically by the inversion analysis method. The empirical formula can provide theoretical guidance for nozzle size design and optimization. The results show that the non-contact 3D laser scanning system is a valuable method for reconstructing the model of the ablated nozzle, and the empirical formula of ablation rate can accurately predict the ablation rate of the nozzle.

3D Imaging with Fringe Projection for Food and Agricultural Applications—A Tutorial

The rising global population, in conjunction with the increasing demand, decreasing labor supply, and increasing costs in the agricultural sector, has induced a need for automation in this industry. Many of these tasks are simplified using depth images and are accomplished using the help of 3D sensing technology such as stereo vision and time of flight methods. While there are various merits to these approaches, there is a need for high-speed, high-accuracy 3D profiling approaches in this rapidly advancing industry. Fringe Projection Profilometry is a variation of structured light technology, which has the advantage of having high speed in the kilohertz range, and sub-millimeter accuracy, which could be extremely beneficial for this sector to adopt. In this article, we seek to provide a tutorial on this technology, explain its various principles along with the basic methodology, and expound on its advantages. We demonstrate some example results using soybean roots and spinach leaves to show its utility, discuss potential reasons as to why this has not yet been widely adopted by this industry, review its potential limitations, and examine possible ways those limitations can be addressed so that they do not present a roadblock in its adoption.

Model-Free Adaptive Impedance Control for Autonomous Robotic Sanding

Sanding is a common yet important task in the manufacturing of many wooden objects (e.g. furniture, decoration box), where the coated layer attached on objects is removed after the interaction with sanding belts. Existing sanding operation is heavily dependent on manual works, which is highly labor-intensive and with the low consistency of quality, and the issues of safety and health also arise after continuous working in the noisy and dusty environment. To deal with the aforementioned, this paper presents the development of a new autonomous sanding robot. The autonomous capability of the developed robot is reflected in the whole procedure of sanding. In particular, the CAD model of the target object is automatically constructed with the structured-light technology, and the sanding behavior on the target surface is self-regulated under the desired impedance model. Such feature makes the robot capable of working towards uncertain objects with minimum human involvement. The proposed impedance controller has the model-free advantage, by using the adaptive neural networks (NNs) to compensate the uncertain dynamics and the unknown disturbances online. The stability of the closed-loop system is rigorously proved with Lyapunov methods, and experimental results on different objects are presented to validate the performance of the developed robot. The implementation of the developed robot can systematically address the problems associated with manual works. Note to Practitioners—The current working environment of sanding is not healthy or safe to humans, due to the nature of noise, dust, high-speed sanding belt. By controlling the robot to autonomously perform sanding tasks can keep humans away from such environment and hence systematically address the issues of health and safety. This paper presents a new impedance control method for sanding robot. In impedance control, the control goal is specified as a dynamic relationship between the contact position and the interaction force. In this paper, the contact position is determined by using the 3D vision sensor, while the interaction force is regulated by referring to the human experience. In addition, the influence caused by unmodeled factors (e.g. unknown dynamics of the sanded object) is also dealt with by using the techniques of NNs. Such setting can effectively guarantee the sanding quality and also avoid the physical damage to the sanded object. Therefore, it lays the foundation for the autonomous robotic sanding.

Structured light field three-dimensional measurement based on equivalent camera array model for highly reflective surfaces

Light-field cameras are gaining attention for their unique light gathering and post-capture processing capabilities. In our previous work, we combined light-field imaging with structured light technology to realize three-dimensional (3D) reconstruction for highly reflective surface. However, due to the lack of effective calibration of the light-field camera and reconstruction systems, it cannot meet the requirements of precise 3D measurement. A universal sub-aperture image extraction algorithm for the light-field camera with hexagonal microlens is presented to accurately extract the multidirectional images. Then, we explore the spatial relationship in equivalent camera array (ECA) of focused light-field camera according to the perspective variation of sub-aperture images. After that, an accurate 3D measurement system based on ECA model is proposed to achieve 3D measurement for highly reflective surface. To test the system’s practical performance, a precision analysis is conducted for a standard gauge block with ground truth. By comparing the reconstruction result of traditional method and ours, we demonstrated the validity of the proposed method to perform 3D measurement for highly reflective surface.

Programmable, Spatiotemporal Control of Colloidal Motion Waves via Structured Light.

Traveling waves in a reaction-diffusion system are essential for long-range communication in living organisms and inspire biomimetic materials of similar capabilities. One recent example is the traveling motion waves among photochemically oscillating, silver (Ag)-containing colloids. Being able to manipulate these colloidal waves holds the key for potential applications. Here, we have discovered that these motion waves can be confined by light patterns and that the chemical clocks of silver particles are moved forward by reducing local light intensity. Using these discoveries as design principles, we have applied structured light technology for the precise and programmable control of colloidal motion waves, including their origins, propagation directions, paths, shapes, annihilation, frequency, and speeds. We have also used the controlled propagation of colloidal waves to guide chemical messages along a predefined path to activate a population of micromotors located far from the signal. Our demonstrated capabilities in manipulating colloidal waves in space and time offer physical insights on their operation and expand their usefulness in the fundamental study of reaction-diffusion processes. Moreover, our findings inspire biomimetic strategies for the directional transport of mass, energy, and information at micro- or even nanoscales.

Hybrid-quality-guided phase fusion model for high dynamic range 3D surface measurement by structured light technology.

Using structured light to measure the 3D shape of a high dynamic range (HDR) surface has been always a challenging problem, and fusion of multi-group images with different exposures is recognized as an effective solution. It tends to select the phase with unsaturated and maximum gray intensity as the final phase, which has two problems: 1) the selection criteria are too simple to fully evaluate the phase quality, and 2) it is affected by the image noise, camera's nonlinear response, local reflection and other factors and the phase with the best quality among the initial phases may also have an error. Aiming to solve these issues, this paper presents a hybrid-quality-guided phase fusion (HPF) model. In this model, a hybrid-quality measure is first proposed to evaluate the phase quality more comprehensively. Then, all initial phases are weighted and fused under the guidance of the hybrid-quality measure to obtain a more accurate phase as the final one. Through this model, a more complete and accurate 3D shape of the HDR surface can be reconstructed, and its validity has been verified by several experiments.

3D reconstruction from structured-light profilometry with dual-path hybrid network

With the rapid development of high-speed image sensors and optical imaging technology, these have effectively promoted the improvement of non-contact 3D shape measurement. Among them, striped structured-light technology has been widely used because of its high measurement accuracy. Compared with classical methods such as Fourier transform profilometry, many deep neural networks are utilized to restore 3D shape from single-shot structured light. In actual engineering deployments, the number of learnable parameters of convolution neural network (CNN) is huge, especially for high-resolution structured-light patterns. To this end, we proposed a dual-path hybrid network based on UNet, which eliminates the deepest convolution layers to reduce the number of learnable parameters, and a swin transformer path is additionally built on the decoder to improve the global perception of this network. The experimental results show that the learnable parameters of the model are reduced by 60% compared with the UNet, and the measurement accuracy is not degraded at the same time. The proposed dual-path hybrid network provides an effective solution for structured-light 3D reconstruction and its practice in engineering.

A novel bone registration method using impression molding and structured-light 3D scanning technology.

Accurate bone registration is critical for computer navigation and robotic surgery. Existing registration systems are expensive, cumbersome, limited in accuracy and/or require intraoperative radiation. We recently reported a novel method of registration utilizing an inexpensive, compact, and X-ray-free structured-light 3D scanner. However, this technique is not always practical in a real surgical setting where soft tissue and blood can obstruct the continuous line-of-sight required for structured-light technology. We sought to remedy these limitations using a novel technique using rapid-setting impression molding to capture bone surface features and scan the undersurface of the mold with a structured-light scanner. The photonegative of this mold is compared to the preoperative computed tomography (CT)-scan to register the bone. A registration accuracy study was conducted on 36 CT-scanned femur sawbones, simulating typical exposure in hip/knee arthroplasty and bone tumor surgery. A cadaver experiment was also conducted to evaluate the feasibility of using the impression molding in a more realistic operating room setting. The registration accuracy of the proposed technique was 0.50 ± 0.19 mm. This was close to the reported accuracy of 0.43 ± 0.18 mm using a structured-light scanner without impression molding (p = 0.085). In comparison, historical values for "paired-point" and intraoperative CT image-based registration methods currently used in modern robotic/computer-navigation systems were 0.68 ± 0.14 mm (p = 0.004) and 0.86 ± 0.38 mm, respectively. The registration accuracy of the cadaver experiment was consistent with that of sawbone experiments. Although future studies are needed to extend to human subjects, this study shows that the impression molding method can produce comparable or better registration accuracy than the existing techniques.

A comparison between stereophotogrammetry and smartphone structured light technology for three-dimensional face scanning

To compare three-dimensional facial scans obtained by stereophotogrammetry with two different applications for smartphone supporting the TrueDepth system, a structured light technology. Facial scans of 40 different subjects were acquired with three different systems. The 3dMDtrio Stereophotogrammetry System (3dMD, Atlanta, Ga) was compared with a smartphone (iPhone Xs; Apple, Cupertino, Calif) equipped with the Bellus3D Face Application (version 1.6.11; Bellus3D Inc, Campbell, Calif) or Capture (version 1.2.5; Standard Cyborg Inc, San Francisco, Calif). Times of image acquisition and elaboration were recorded. The surface-to-surface deviation and the distance between 18 landmarks from 3dMD reference images to those acquired with Bellus3D or Capture were measured. Capturing and processing times with the smartphone applications were considerably longer than with the 3dMD system. The surface-to-surface deviation analysis between the Bellus3D and 3dMD showed an overlap percentage of 80.01% ± 5.92% and 56.62% ± 7.65% within the ranges of 1 mm and 0.5 mm discrepancy, respectively. Images from Capture showed an overlap percentage of 81.40% ± 9.59% and 56.45% ± 11.62% within the ranges of 1 mm and 0.5 mm, respectively. The face image acquisition with the 3dMD device is fast and accurate, but bulky and expensive. The new smartphone applications combined with the TrueDepth sensors show promising results. They need more accuracy from the operator and more compliance from the patient because of the increased acquisition time. Their greatest advantages are related to cost and portability.

A COMPARISON OF LASER AND STRUCTURED LIGHT SCANNING TECHNOLOGIES FOR ARCHAEOLOGICAL APPLICATIONS

Reverse engineering and in particular three-dimensional digitization have become an essential part of the documentation of archaeological findings. 3D scanning produces a high-precision digital reference document. The factors that influence the quality of the 3D scanned data are the scanned object’s surface colour, its glossiness and geometry, and the ambient light during the scanning process. However, the actual equipment and scanning technologies are of primary importance. The current paper presents a qualitative and quantitative comparison between two 3D scanning devices of different technologies; structured light 3D scanning and laser 3D scanning. The benchmark for this comparison is an ancient Roman vase from the city of Thessaloniki, Greece. The object was scanned with every possible setting on each scanner, but only one configuration of settings on each device was selected for the final comparison. The main criterion for the final selection of the two 3D models acquired with the use of the two technologies was the proximity in the number of points and polygons produced for digitally restoring the ancient vase in the best possible way. The results indicate important differences regarding the accuracy of the final digital model. The laser technology produced better accuracy but with a significant cost in scanning time and model data size. On the other hand, the structured light technology achieved the optimal combination of scanning quality and accuracy, along with reduced acquisition time of scan data.

A Generative and Entropy-Based Registration Approach for the Reassembly of Ancient Inscriptions

An increased interest in computer-aided heritage reconstruction has emerged in recent years due to the maturity of sophisticated computer vision techniques. Concretely, feature-based matching methods have been conducted to reassemble heritage assets, yielding plausible results for data that contains enough salient points for matching. However, they fail to register ancient artifacts that have been badly deteriorated over the years. In particular, for monochromatic incomplete data, such as 3D sunk relief eroded decorations, damaged drawings, and ancient inscriptions. The main issue lies in the lack of regions of interest and poor quality of the data, which prevent feature-based algorithms from estimating distinctive descriptors. This paper addresses the reassembly of damaged decorations by deploying a Generative Adversarial Network (GAN) to predict the continuing decoration traces of broken heritage fragments. By extending the texture information of broken counterpart fragments, it is demonstrated that registration methods are now able to find mutual characteristics that allow for accurate optimal rigid transformation estimation for fragments alignment. This work steps away from feature-based approaches, hence employing Mutual Information (MI) as a similarity metric to estimate an alignment transformation. Moreover, high-resolution geometry and imagery are combined to cope with the fragility and severe damage of heritage fragments. Therefore, the testing data is composed of a set of ancient Egyptian decorated broken fragments recorded through 3D remote sensing techniques. More specifically, structured light technology for mesh models creation, as well as orthophotos, upon which digital drawings are created. Even though this study is restricted to Egyptian artifacts, the workflow can be applied to reconstruct different types of decoration patterns in the cultural heritage domain.

LED source interferometer for microscopic fringe projection profilometry using a Gates’ interferometer configuration

The fringe projection technique is considered one of the most reliable solutions for structured light technology, developed for non-contact three-dimensional (3D) data acquisition. When working with this technique, applying it on a microscopic scale, one often encounters some problems such as the limited depth of field of the projected fringe pattern, typical of the imaging lenses, or the formation of speckle-noise when laser interferometric fringes are projected, which are problems that can cause significant errors on test surfaces whose height exceeds the depth of field, and on which, the other hand, speckle noise affects the resolution of the technique. This paper presents a Gates’ interferometer configuration with a LED source to project a fringe pattern without speckle noise and a very long field depth. We explain how a non-located sinusoidal fringe pattern can be obtained using two-beamsplitter cubes and a LED light source. Moreover, we compare measurements made using the LED and laser concerning a commercial contact profilometer.

Research on RGB-D image recognition technology based on feature fusion and machine learning

The three-dimensional RGB-D image contains not only the color and texture information of the two-dimensional image, but also contains the surface geometry information of the target. This article analyzes the RGB-D image recognition methods, including stereo vision technology, structured light technology, etc. By studying the application points of RGB-D image recognition technology under the background of feature fusion and machine learning, the purpose is to improve the richness of image recognition content and provide reference for the smooth development of the follow-up work.