Coaxial Imaging Research Articles

In situ porosity intelligent classification of selective laser melting based on coaxial monitoring and image processing

Selective laser melting (SLM) has shown unique advantages in fabricating metal components. However, the part quality still largely suffered from the porosity defects that are not easily detected and eliminated. In this work, the objective is to realize the porosity classification based on high-speed melt pool images. A coaxial high-speed in situ monitoring system was first developed to capture the melt pool images during the multi-track and multi-layer printing process. Then, a novel image and data processing method was proposed to extract the critical and high-level melt pool features data. Three intelligent machine learning algorithms of back propagation neural network (BPNN), support vector machine (SVM), and deep belief network (DBN) were finally developed to match the features data with porosity modes. Results show that it is feasible and effective for the proposed method to realize porosity classification during the SLM process, which can provide a potential to reduce porosity defects.

In-line monitoring of focus shift by kerf width detection with coaxial thermal imaging during laser cutting

In-line monitoring of focus shift by kerf width detection with coaxial thermal imaging during laser cutting

Simultaneous Burr and Cut Interruption Detection during Laser Cutting with Neural Networks.

In this contribution, we compare basic neural networks with convolutional neural networks for cut failure classification during fiber laser cutting. The experiments are performed by cutting thin electrical sheets with a 500 W single-mode fiber laser while taking coaxial camera images for the classification. The quality is grouped in the categories good cut, cuts with burr formation and cut interruptions. Indeed, our results reveal that both cut failures can be detected with one system. Independent of the neural network design and size, a minimum classification accuracy of 92.8% is achieved, which could be increased with more complex networks to 95.8%. Thus, convolutional neural networks reveal a slight performance advantage over basic neural networks, which yet is accompanied by a higher calculation time, which nevertheless is still below 2 ms. In a separated examination, cut interruptions can be detected with much higher accuracy as compared to burr formation. Overall, the results reveal the possibility to detect burr formations and cut interruptions during laser cutting simultaneously with high accuracy, as being desirable for industrial applications.

Non-contact local conductivity measurement of metallic nanowires based on semi-near-field reflection of microwave atomic force microscopy

In this study, a non-contact and quantitative evaluation method was developed to measure the conductivity of metallic nanowires at nanometer-scale resolution. Using a coaxial probe, microwave images and topographical images were simultaneously obtained for three nanowires via microwave atomic force microscopy (M-AFM). A semi-near-field model was established to describe the distribution of the electric field between the probe and the sample. Based on this model, the local conductivities of metallic nanowires on the nanometer scale were quantitatively evaluated in a single scan, using a metal strip substrate to calibrate the reflected signal.

Fast calibration method of alignment prism based on image transformation relation

The alignment assembly based on the principle of coaxial imaging has high accuracy, and the accuracy of coaxial imaging is the key to it. Manually calibrating the pose of the prism to ensure the accuracy of coaxial imaging, which has disadvantages such as difficult operation and long time consuming. This paper analyzes the image relation between the upper and lower parts of the prism based on the optical imaging theory, and solves the image transformation relation between the target part and the assembly part with an experimental method, and proposes a rapid calibration method for the alignment prism based on the image transformation relation. The tedious steps of manually adjusting the prism are eliminated. The experimental results show that this method realizes the high-precision assembly of parts by the micro-assembly system.

Optimization method using nodal aberration theory for coaxial imaging systems with radial basis functions based on surface slope

The radial basis functions based on the surface slope (RBF-Slope) freeform surfaces model has demonstrated stronger fitting ability and better optical performance than the conventional RBF model. However, the large number of basis functions and optimization variables of the RBF-Slope model may result in convergence problems during optimization for optical systems consisting of freeform surfaces characterized by RBF-Slope. To overcome these drawbacks, we use Zernike polynomials to link the RBF-Slope model to aberration correction and propose a new optimization method for coaxial imaging systems using the RBF-Slope model based on nodal aberration theory (NAT). The aberrations generated by the conic parameter and Zernike terms up to Z 17 / 18 of the Zernike freeform surface at the non-stop surface in the coaxial imaging system are analyzed, and the gradient descent is implemented to obtain the optimal coefficients of the Zernike surface, which is then fitted by the RBF-Slope surface for further optimization. The method is applied to the optimization of a secondary mirror using the RBF-Slope model in a two-mirror telescope and proved to have better results than traditional commonly used direct optimization. This research offers an important reference for optimization using NAT and provides valuable insight into the optimization method for RBF-Slope freeform surfaces.

A virtual sensing approach for monitoring melt-pool dimensions using high speed coaxial imaging during laser powder bed fusion of metals

Abstract Metal parts produced by Laser Powder Bed Fusion (L-PBF) are frequently used for demanding applications. To meet stringent safety and certification requirements, a better understanding of melt-pool behavior and stability during processing is desired. This work presents a novel, fast and economically feasible virtual sensing approach for accurate estimation of melt-pool depth and width during L-PBF of metals. In a first step, the melt-pool width is determined by GPU-based processing of images from a high-speed coaxial camera monitoring system. In a second step, a physics-based analytical model is used to calculate the melt-pool depth-to-width ratio from the processing conditions and material properties. In a third and last step, the results from the first two steps are combined to estimate the melt-pool depth. Experimental validation of these predicted melt-pool dimensions is performed on 316L SS single layer strips that are consecutively produced, cross-sectioned, polished and etched to reveal the actual melt-pool boundaries. The results indicate an average relative error on the predicted melt-pool depth of 9.9% and 2.8% for the full L-PBF parameter range and for the optimal parameter range respectively. This gives confidence in the predictive capabilities of a virtual sensing approach using coaxial camera images for the assessment of the melt-pool depth and process stability.

Coaxial Monitoring of AISI 316L Thin Walls Fabricated by Direct Metal Laser Deposition.

Direct metal laser deposition (DMLD) is an additive manufacturing technique suitable for coating and repair, which has been gaining a growing interest in 3D manufacturing applications in recent years. However, its diffusion in the manufacturing industry is still limited due to technical challenges to be solved—both the sub-optimal quality of the final parts and the low repeatability of the process make the DMLD inadequate for high-value applications requiring high-performance standards. Thus, real-time monitoring and process control are indispensable requirements for improving the DMLD process. The aim of this study was the optimization of deposition strategies for the fabrication of thin walls in AISI 316L stainless steel. For this purpose, a coaxial monitoring system and image processing algorithms were employed to study the melt pool geometry. The comparison tests carried out highlighted how the region-based active contour algorithm used for image processing is more efficient and stable than others covered in the literature. The results allowed the identification of the best deposition strategy. Therefore, it is shown how this monitoring methodology proved to be suitable for designing and implementing the right building strategy for DMLD manufactured 3D components. A fast and stable image processing method was achieved, which can be considered for future closed-loop monitoring in real-time applications.

Coaxial Scanning Three-dimensional Imaging Based on SPAD Array

Coaxial Scanning Three-dimensional Imaging Based on SPAD Array

Observation of Clinical Efficacy of Parylene‐Coated Occluder Based on Electronic Image of Visual Sensor in the Treatment of Congenital Heart Disease

With the acceleration of people’s life rhythm, the incidence of congenital heart disease is getting higher and higher. This study mainly explores the clinical efficacy observation of parylene‐coated occluders based on the electronic image of the visual sensor in the treatment of congenital heart disease. The patient takes a supine position and observes whether the target structure (left ventricular cavity) information is separated from the capture area during rotation, and the angle of the probe is adjusted to ensure that as much target structure information as possible can be obtained during the two‐dimensional image acquisition. During the three‐dimensional ultrasound sampling process, a total of 60 coaxial images were collected within the range of 180°. To reflect the effect of virtual reality, it is necessary to provide users with interactive operation tools. This research provides two methods, mouse interaction and keyboard interaction. Through these two methods, the effect of the basic virtual endoscope is realized. Xtion’s image registration process is completed inside its PS1080 system‐level chip, and all sensor information, color images, and depth images are transmitted to the user host through the USB interface. For example, the point of view can be placed in the heart, because the simulation occlusion must first find a suitable section to better observe and judge the occlusion effect. Therefore, this study uses the mouse interaction method to change the cutting by dragging and rotating the mouse. In the process of releasing the occluder, the parylene‐coated occluder can be released after the X‐ray and ultrasound examination results are satisfactory, and the delivery long sheath and catheter can be sprinkled and compressed to stop the bleeding. During the operation, the ASD measured by two‐dimensional ultrasound after balloon inflation was 24.17 ± 7.94 mm, and the anterior and posterior diameter was 21.30 ± 6.54 mm. This research is helpful for the qualitative and quantitative diagnosis of congenital heart disease.

Optical emission sensing for laser-based additive manufacturing—What are we actually measuring?

Numerous sensing modalities have been utilized to monitor metal additive manufacturing, thus assessing process stability and build quality. One common directed energy deposition sensing method is coaxial “melt pool” imaging, wherein a camera mounted coaxially with the laser-focusing optics views the laser-interaction zone via a dichroic mirror. This work demonstrates that coaxial “melt pool” imaging does not always provide reliable, as-solidified, pool geometry measurements. First, coaxially acquired images captured using a charge injection device camera for a range of laser powers, translation speeds, and powder flow rates are compared to optical profilometry-measured true deposition geometries. Furthermore, coaxial “melt pool” images and plume geometries, measured using a bandpass filtered CCD camera, show significant, abrupt “melt pool” geometry changes, which correspond to plume geometry changes. Additionally, interbuild comparisons demonstrate a linear relationship between “melt pool” and plume geometry. The authors conclude that coaxial imaging, as commonly implemented, is an often unreliable melt pool geometry measurement because plume emissions in and around the laser-interaction zone may obscure the melt pool.

Coaxial spectroscopic imaging ellipsometry for volumetric thickness measurement.

We propose spectroscopic imaging ellipsometry that can measure spectral ellipsometric signals in the entire field of view simultaneously without areal scanning or operation of polarization devices. The proposed imaging ellipsometry is configured in a coaxial optical structure so that the high magnification objective lens is applicable and the spatial resolution is highly increased. Without the operation of polarization components and to efficiently obtain the spectral data in the object plane, the ellipsometric parameters are encoded into the high frequency in the spectral domain and are measured by an imaging Michelson interferometer. The volumetric thickness measurement by the proposed method was verified by comparing the thickness results of the SiO2/Si sample that has four different thicknesses with commercial ellipsometer results.

Investigation on coaxial visual characteristics of molten pool in laser-based directed energy deposition of AISI 316L steel

Abstract Laser-based directed energy deposition (LDED), an additive manufacturing technique, has received increasing attention from the industry and academia. Unstable reliability and poor repeatability in the production of metal parts are two challenges currently faced by LDED. In-situ monitoring and adaptive control technology based on visual characteristics serves as effective methods to ensure part quality and process stability. Although molten pools directly influence metallurgical behaviors in LDED, their visual characteristics still require further investigation. In this work, a coaxial visual image of a molten pool was captured during the LDED of 316 L material and divided into three regions. Then, the evolutionary behaviors of different regions during single-layer single-track processing, multi-layer one-track stacking, and one-layer multi-track overlapping were analyzed. Additionally, the metallurgical behaviors of the molten pool, including the formation mechanisms of slag characteristics, were explicated. Results showed that molten pool characteristics could be presented and described with visual images reasonably. These characteristics could indicate the state of melting, heat conduction and convection, and overlapping position. Meanwhile, coaxial visual characteristics were closely related to product qualities. This work could enhance the understanding of LDED process mechanisms and improve real-time monitoring capability. The proposed approach for the determination of molten pool characteristics is expected to be applicable to other materials.

Coaxiality Evaluation of Coaxial Imaging System with Concentric Silicon-Glass Hybrid Lens for Thermal and Color Imaging.

Thermal imaging is useful for tasks such as detecting the presence of humans and recognizing surrounding objects in the operation of several types of robots, including service robots and personal mobility robots, which assist humans. Because the number of pixels on a thermal imager is generally smaller than that on a color imager, thermal images are more useful when combined with color images, assuming that the correspondence between points in the images captured by the two sensors is known. In the literature, several types of coaxial imaging systems have been reported that can capture thermal and color images, simultaneously, from the same point of view with the same optical axis. Among them, a coaxial imaging system using a concentric silicon–glass hybrid lens was devised. Long-wavelength infrared and visible light was focused using the hybrid lens. The focused light was subsequently split using a silicon plate. Separate thermal and color images were then captured using thermal and color imagers, respectively. However, a coaxiality evaluation of the hybrid lens has not been shown. This report proposes an implementation and coaxiality evaluation for a compact coaxial imaging system incorporating the hybrid lens. The coaxiality of the system was experimentally demonstrated by estimating the intrinsic and extrinsic parameters of the thermal and color imagers and performing 2D mapping between the thermal images and color images.

Feasibility and Safety of a Coaxial Dual-Wavelength Optical Coherence Tomography Apparatus

Purpose: To evaluate the feasibility and safety of a coaxial dual-wavelength optical coherence tomography (OCT) device (marked as Hydra-OCT). Methods: Healthy participants without ocular pathology underwent retinal imaging using the Hydra-OCT allowing for simultaneous measurement of retinal scanning of 840 and 1,072 nm wavelength. Before and after measurement, best-corrected visual acuity and patients’ comfort were assessed. Representative OCT images from both wavelengths were compared by 5 independent graders using a subjective grading scheme. Results: A total of 30 eyes of 30 participants (8 females and 22 males) with a mean age of 26.5 years (range from 19 to 55 years) were included. Dual-wavelength image acquisition was made possible in each subject. The participant’s effort and comfort assessment using the Hydra-OCT imaging revealed an equivalent value as compared to the commercially available OCT machine. No adverse events were reported, and visual acuity was not altered by the Hydra-OCT. Imaging between the systems was comparable. Conclusions: This study provides evidence for the feasibility and safety of a coaxial dual-wavelength OCT imaging method under real-life conditions. The novel Hydra-OCT imaging device may offer additional insights into the pathology of retinal and choroidal diseases.

In-process measurement of melt pool cross-sectional geometry and grain orientation in a laser directed energy deposition additive manufacturing process

Understanding the behaviour of melt pool during laser directed energy deposition (L-DED) is essential for the prediction and control of process quality. Previous effort was focused on the observation of melt pool surface characteristics. In this paper, a coaxial imaging system was employed to determine the melt pool cross sectional geometry and to predict solidified grain orientation during a high deposition rate L-DED process. The image processing procedure, deposition track cross-sectional profile prediction and the relationship between melt pool shape and melt pool dynamics, and grain growth orientation were investigated. Results show that sharp melt pool edges can be obtained so that melt pool width can be predicted with an accuracy of more than 95%. The estimation method of melt pool length has an accuracy of 90%. With the experimental melt pool width and depth data, the cross-sectional profiles of deposited track are predicted at an accuracy of 92% and a good match with experimental data is obtained. The melt pool formation is found to be able to allow the prediction of crystal growth directions during solidification.

透射共轴式光声消化内窥成像系统

透射共轴式光声消化内窥成像系统

Depth Sensing by Near-Infrared Light Absorption in Water.

This paper introduces a novel depth recovery method based on light absorption in water. Water absorbs light at almost all wavelengths whose absorption coefficient is related to the wavelength. Based on the Beer-Lambert model, we introduce a bispectral depth recovery method that leverages the light absorption difference between two near-infrared wavelengths captured with a distant point source and orthographic cameras. Through extensive analysis, we show that accurate depth can be recovered irrespective of the surface texture and reflectance, and introduce algorithms to correct for nonidealities of a practical implementation including tilted light source and camera placement, nonideal bandpass filters and the perspective effect of the camera with a diverging point light source. We construct a coaxial bispectral depth imaging system using low-cost off-the-shelf hardware and demonstrate its use for recovering the shapes of complex and dynamic objects in water. We also present a trispectral variant to further improve robustness to extremely challenging surface reflectance. Experimental results validate the theory and practical implementation of this novel depth recovery paradigm, which we refer to as shape from water.

Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks

In this paper, an innovative monitoring system capable of diagnosing the penetration state during the laser welding process is introduced, which consists of two main blocks: a coaxial visual monitoring platform and a penetration state diagnosis unit. The platform can capture coaxial images of the interaction zone during the laser welding through a partially transmitting mirror and a high-speed camera. An image dataset representing four welding states was created for training and validation. The unit mainly consists of an embedded power-efficient computing TX2 and image processing algorithms based on a convolution neural network (CNN). Experiment results show that the platform can stably capture state-of-the-art welding images. The CNN used for a diagnosis of the penetration state is optimized using an optimal network structure and hyperparameters, applying a super-Gaussian function to initialize the weights of the convolutional layer. Its latency on TX2 is less than 2 ms, satisfying the real-time requirement. During the real laser welding of tailor-rolled blanks, a penetration state diagnosis with an accuracy of 94.6 % can be achieved even if the illumination changes significantly. The similar accuracy between the validating set and a real laser welding demonstrates that the proposed monitoring system has strong robustness. The precision and recall ratios of the CNN are higher than those of other methods such as a histogram of oriented gradients and local binary pattern.

Large-scale volumetric flow visualization of the unsteady wake of a flapping-wing micro air vehicle

The objective of this experimental investigation is the volumetric visualization of the near wake topology of the vortex structures generated by a flapping-wing micro air vehicle. To achieve the required visualization domain (which in the present experiments amounts to a size of 60,000 cm3), use is made of robotic particle image velocimetry, which implements coaxial illumination and imaging in combination with the use of helium-filled soap bubbles as tracer particles. Particle trajectories are determined via Lagrangian particle tracking and information of different phases throughout the flapping cycle is obtained by means of a phase-averaging procedure applied to the particle tracks. Experiments have been performed at different settings (flow speed, flapping frequency, and body angle) that are representative of actual flight conditions, and the effect of reduced frequency on the wake topology is investigated. Furthermore, experiments have been carried out in both tethered and free-flight conditions, allowing an unprecedented comparison between the aerodynamics of the two conditions.Graphic abstract