Industrial Inspection Applications Research Articles

A calibration method of line-structured light system for measuring outer circle dimension during machining

The line-structured light system is widely utilized for various 3D profile measurements due to its convenience and high efficiency. However, the existing calibration methods primarily focus on establishing the relationship between object points and image points, which imposes stringent requirements on the calibration target. To address this issue, a practical calibration method based on interpolation function is proposed in order to measure a specific 3D profile - namely, the diameter of a rotating workpiece during machining. Firstly, the light stripe projected onto an ordinary machined workpiece by a light plane is captured using a Charge-Coupled Device (CCD) camera and subsequently fitted to a quadratic elliptic curve after undergoing image processing. Subsequently, the cubic spline interpolation function is selected to directly establish a mapping relationship between geometric characteristics of the elliptic curve and the diameter measurement. Several experiments are conducted to evaluate the proposed methods. The experimental results demonstrate that compared with two comparison methods, our approach reduces the calibration error of linear structured light systems by 69 % and 51 %, respectively; furthermore, it enables restoration of smooth and detailed 3D models through parallel movement of the system. Moreover, our entire calibration process proves practicable in simplifying experimental procedures while remaining suitable for industrial inspection applications.

Progress in Advanced Infrared Optoelectronic Sensors.

Infrared optoelectronic sensors have attracted considerable research interest over the past few decades due to their wide-ranging applications in military, healthcare, environmental monitoring, industrial inspection, and human-computer interaction systems. A comprehensive understanding of infrared optoelectronic sensors is of great importance for achieving their future optimization. This paper comprehensively reviews the recent advancements in infrared optoelectronic sensors. Firstly, their working mechanisms are elucidated. Then, the key metrics for evaluating an infrared optoelectronic sensor are introduced. Subsequently, an overview of promising materials and nanostructures for high-performance infrared optoelectronic sensors, along with the performances of state-of-the-art devices, is presented. Finally, the challenges facing infrared optoelectronic sensors are posed, and some perspectives for the optimization of infrared optoelectronic sensors are discussed, thereby paving the way for the development of future infrared optoelectronic sensors.

Multi-focus manipulation system based on SNES aberration self-calibration

Abstract Wavefront shaping using digital micromirror devices (DMDs) allows inertia-free focus manipulation with numerous modulation modes and high refresh rates. However, the aberration caused by the curvature of DMD affects the focusing performance. Here, we propose an aberration self-calibration method based on separable natural evolution strategies (SNES). This method searches optimal Zernike coefficients of aberration globally and completes compensation using super-pixel encoding. Compared to the genetic algorithm method, we improve the speed by 62% and achieve better-focused spots. It enables simultaneous scanning of 25 independent focal spots. This advancement supports wavefront shaping applications in optical imaging, industrial inspection, and laser processing.

Deep Learning for Automated Visual Inspection in Manufacturing and Maintenance: A Survey of Open- Access Papers

Quality assessment in industrial applications is often carried out through visual inspection, usually performed or supported by human domain experts. However, the manual visual inspection of processes and products is error-prone and expensive. It is therefore not surprising that the automation of visual inspection in manufacturing and maintenance is heavily researched and discussed. The use of artificial intelligence as an approach to visual inspection in industrial applications has been considered for decades. Recent successes, driven by advances in deep learning, present a possible paradigm shift and have the potential to facilitate automated visual inspection, even under complex environmental conditions. For this reason, we explore the question of to what extent deep learning is already being used in the field of automated visual inspection and which potential improvements to the state of the art could be realized utilizing concepts from academic research. By conducting an extensive review of the openly accessible literature, we provide an overview of proposed and in-use deep-learning models presented in recent years. Our survey consists of 196 open-access publications, of which 31.7% are manufacturing use cases and 68.3% are maintenance use cases. Furthermore, the survey also shows that the majority of the models currently in use are based on convolutional neural networks, the current de facto standard for image classification, object recognition, or object segmentation tasks. Nevertheless, we see the emergence of vision transformer models that seem to outperform convolutional neural networks but require more resources, which also opens up new research opportunities for the future. Another finding is that in 97% of the publications, the authors use supervised learning techniques to train their models. However, with the median dataset size consisting of 2500 samples, deep-learning models cannot be trained from scratch, so it would be beneficial to use other training paradigms, such as self-supervised learning. In addition, we identified a gap of approximately three years between approaches from deep-learning-based computer vision being published and their introduction in industrial visual inspection applications. Based on our findings, we additionally discuss potential future developments in the area of automated visual inspection.

Plasmonic photoconductive terahertz focal-plane array with pixel super-resolution

AbstractImaging systems operating in the terahertz part of the electromagnetic spectrum are attractive due to their ability to penetrate many opaque materials and provide unique spectral signatures of various chemicals. However, the use of terahertz imagers in real-world applications has been limited by the slow speed, large size, high cost and complexity of present systems, largely due to the lack of suitable terahertz focal-plane array detectors. Here we report a terahertz focal-plane array that can directly provide the spatial amplitude and phase distributions, along with the ultrafast temporal and spectral information of an imaged object. It consists of a two-dimensional array of ~0.3 million plasmonic photoconductive nanoantennas optimized to rapidly detect broadband terahertz radiation with a high signal-to-noise ratio. We utilized the multispectral nature of the amplitude and phase data captured by these plasmonic nanoantennas to image different objects, including super-resolved etched patterns in a silicon substrate and defects in battery electrodes. By eliminating the need for raster scanning and spatial terahertz modulation, our terahertz focal-plane array offers more than a 1,000-fold increase in the imaging speed compared with the state of the art and potentially suits a broad range of applications in industrial inspection, security screening and medical diagnosis, among others.

High-Confidentiality X-Ray Imaging Encryption Using Prolonged Imperceptible Radioluminescence Memory Scintillators.

X-ray imaging plays an increasingly crucial role in clinical radiography, industrial inspection, and military applications. However, current X-ray imaging technologies have difficulty in protecting against information leakage caused by brute force attacks via trial-and-error. Here high-confidentiality X-ray imaging encryption by fabricating ultralong radioluminescence memory films composed of lanthanide-activated nanoscintillators (NaLuF4 : Gd3+ or Ce3+ ) with imperceptible purely-ultraviolet (UV) emission is reported. Mechanistic investigations unveil that ultralong X-ray memory is attributed to the long-lived trapping of thermalized charge carriers within Frenkel defect states and subsequent slow release in the form of imperceptible radioluminescence. The encrypted X-ray imaging can be securely stored in the memory film for more than 7 days and optically decoded by perovskite nanocrystal. Importantly, this encryption strategy can protect X-ray imaging information against brute force trial-and-error attacks through the perception of lifetime change in the persistent radioluminescence. It is further demonstrated that the as-fabricated flexible memory film enables achieving of 3D X-ray imaging encryption of curved objects with a high spatial resolution of 20 lp/mm and excellent recyclability. This study provides valuable insights into the fundamental understanding of X-ray-to-UV conversion in nanocrystal lattices and opens up a new avenue toward the development of high-confidential 3D X-ray imaging encryption technologies.

Single-pixel real-part and magnitude imaging system based on digital micromirror device

Single-pixel imaging (SPI) can capture images using a single-pixel detector. However, conventional SPI schemes only provide a magnitude image of the object, where the phase information is completely lost. Here, we present a dual-modal SPI (DMSPI) system capable of simultaneously capturing the real-part and magnitude images. Since the real-part image fuses the phase distribution, DMSPI has a strong information acquisition capability for all types of objects. The DMSPI system utilizes the two reflection arms provided by the digital micromirror device (DMD) to perform zero-frequency detection in one arm and bucket detection in the other to achieve dual-modal imaging. Benefiting from the unique modulation characteristics of DMD, the DMSPI system is simple and efficient, with high spatial resolution and fast imaging speed. Thus, it might find broad applications in biomedical diagnostics and industrial inspection.

Real-Time Defect Detection Model in Industrial Environment Based on Lightweight Deep Learning Network

Surface defect detection in industrial environments is crucial for quality management and has significant research value. General detection networks, such as the YOLO series, have proven effective in various dataset detections. However, due to the complex and varied surface defects of industrial products, many defects occupy a small proportion of the surface and fall into the category of typical small target detection problems. Moreover, the complexity of general detection network architectures relies on high-tech hardware, making it difficult to deploy on devices without GPUs or on edge computing and mobile devices. To meet the practical needs of industrial product defect inspection applications, this paper proposes a lightweight network specifically designed for defect detection in industrial fields. This network is composed of four parts: a backbone network, a multiscale feature aggregation network, a residual enhancement network, and an attention enhancement network. The network includes a backbone network that integrates attention layers for feature extraction, a multiscale feature aggregation network for semantic information, a residual enhancement network for spatial focus, and an attention enhancement network for global–local feature interaction. These components enhance detection performance for diverse defects while maintaining low hardware requirements. Experimental results show that this network outperforms the latest and most popular YOLOv5n and YOLOv8n models in the five indicators P, R, F1, mAP@.5, and GFLOPS when used on four public datasets. It even approaches or surpasses the YOLOv8s and YOLOv5s models with several times the GFLOPS computation. It balances the requirements of lightweight real-time and accuracy in the scenario of industrial product surface defect detection.

X-ray Detectors Based on Ga2O3 Microwires.

X-ray detectors have numerous applications in medical imaging, industrial inspection, and crystal structure analysis. Gallium oxide (Ga2O3) shows potential as a material for high-performance X-ray detectors due to its wide bandgap, relatively high mass attenuation coefficient, and resistance to radiation damage. In this study, we present Sn-doped Ga2O3 microwire detectors for solar-blind and X-ray detection. The developed detectors exhibit a switching ratio of 1.66 × 102 under X-ray irradiation and can operate stably from room temperature to 623 K, which is one of the highest reported operating temperatures for Ga2O3 X-ray detectors to date. These findings offer a promising new direction for the design of Ga2O3-based X-ray detectors.

Low dark current and high stability X-ray detector based on FAPbI3/Ga2O3 heterojunction

X-ray detectors have drawn great attention due to their wide applications in medical imaging and industrial inspection. Lead halide perovskite exhibits excellent X-ray detection merits because of its high-Z element and outstanding photoelectric properties. However, the perovskite-based X-ray detectors exhibit poor stability, low resistivity and large dark leakage current due to the soft lattice and ion migration of perovskite materials. Herein, we used Ga2O3 compact layer to form a heterojunction structure with formamidinum lead iodide (FAPbI3) to largely ease these issues. Compared with the bare FAPbI3 detector, the dark current of the FAPbI3/Ga2O3 heterojunction detector is reduced by 3 orders of magnitude and the photo-to-dark current ratio (PDCR) increased by 31-fold. The detection limit of 0.23 mGys−1 is obtained which is 20 times lower than the FAPbI3 detector and short response time of 20 ms is obtained. More importantly, the FAPbI3/Ga2O3 device exhibits better repeatability due to the suppression of ion migration by Ga2O3. This work provides an effective strategy to obtain low dark current and stable perovskite-based photodetector.

Flexible Multi-Element Photothermoelectric Detectors Based on Spray-Coated Graphene/Polyethylenimine Composites for Nondestructive Testing

Photothermoelectric (PTE) detectors receive much attention owing to the superiority of self-powered, non-bias input, and friendly ambient environments, facilitating abundant prospective applications in industrial inspection, medical diagnostics, homeland security, and wearable Internet of Things. However, many drawbacks of currently applicable PTE materials, involving unstable material oxidation, an uncontrollable fabrication process, and unscalable manufacturing, hinder the development of industrial productions. Herein, we demonstrate a vertical graphene/polyethylenimine composite PTE detector fabricated with an optimized spray-coating method in compact alignment on various surfaces, achieving a significant photovoltage detectivity and responsivity of 6.05 × 107 cm Hz1/2 W-1 and 2.7 V W-1 response at a 973 K blackbody temperature radiation (2.98 μm peak wavelength). In addition, the long-term stability and resistible concave and convex bending flexibility are presented. Furthermore, a nondestructive testing system is established and verified through high-spatial-resolution and high-penetration illustration. Overall, the spray-coated and flexible PTE graphene/polyethylenimine multi-elements with broadband infrared absorption compatibility and stable energy conversion are promising candidates for future health monitoring and wearable electronics.

MFCP-Net: Multiscale Feature Correlation Perception Network for Image Segmentation Based on Background Prior

Due to their exceptional capacity to extract features, methods based on deep learning have demonstrated good performance in industrial vision inspection applications. However, the generalizability and robustness of inspection models are challenged by factors such as insufficient data, great differences in data types, and complex illumination. To address these problems, multiscale feature correlation perception network (MFCP-Net), a semantic segmentation model with variable topology, is proposed to segment images with prior background. MFCP-Net uses a Siamese structure with shared weights to extract the multiscale features of the template and test images. A feature correlation perception (FCP) block is designed to measure the correlation of latent representation between the template and test images at multiscales, using background information more effectively. In addition, MFCP-Net provides different topological structures for the training and testing phases. A background auto-correlation branch is included during the testing phase to eliminate false positives observed in the background. MFCP-Net was compared with the state-of-the-art models on the proposed automobile instrument detection system, demonstrating that it offers superior accuracy and generalizability, especially in the case of insufficient data and big variances across data.

Sub-Diffraction Focusing Using Metamaterial-Based Terahertz Super-Oscillatory Lens

This paper presents a metamaterial-based super-oscillatory lens (SOL) fabricated by photolithography on a glass substrate and designed to operate at sub-terahertz (sub-THz) frequencies. The lens consists of repeating crisscross patterns of five-ring slits with sub-wavelength diameter. The lens is capable of generating multiple focal points smaller than the diffraction limit, thereby allowing many points to be inspected simultaneously with sub-wavelength resolution. After elucidating the influence of the lens parameters on light collection through calculations by the finite element method, the fabricated lens was then evaluated through actual experiments and found to have a focal length of 7.5 mm (2.5λ) and a hot spot size of 2.01 mm (0.67λ) at 0.1 THz (λ = 3 mm), which is 0.27 times the diffraction limit of the lens. This demonstrated sub-diffraction focusing capability is highly effective for industrial inspection applications utilizing terahertz waves.

A fully actuated aerial manipulator system for industrial contact inspection applications

PurposeThis paper aims to present a fully actuated aerial manipulator (AM) with a robust motion/force hybrid controller for conducting contact-typed inspection tasks in industrial plants.Design/methodology/approachAn AM is designed based on a hexarotor with tilted rotors and a rigidly attached end effector. By tilting the rotors, the position and attitude of the AM can be controlled independently, and the AM can actively exert forces on industrial facilities through the rigidly attached end effector. A motion/force hybrid controller is proposed to perform contact-typed inspection tasks. The contact-typed inspection task is divided into the approach phase and the contact phase. In the approach phase, the AM automatically approaches the contact surface. In the contact phase, a motion/force hybrid controller is used for contact-typed inspection. Finally, a disturbance observer (DOB) is used to estimate external disturbances and used as feedforward compensation.FindingsThe proposed AM can slowly approach the contact surface without significant impact in the contact phase. It can realize constant force control in the direction normal to the contact surface in the contact phase, whereas the motion of the remaining directions can be controlled by the operator. The use of the DOB ensures the robustness of the AM in the presence of external wind disturbances.Originality/valueA fully actuated AM system with a robust motion/force hybrid controller is proposed. The effectiveness of the proposed AM system for conducting contact-typed industrial inspection tasks is validated by practical experiments.

CASPPNet: a chained atrous spatial pyramid pooling network for steel defect detection

Automatic machine vision-based defect detection has been successfully applied to many industrial visual inspection applications. However, automatic steel surface defect detection is still a challenging task due to diverse defect categories, low-contrast between defect and complex texture background. To address these challenges, a chained atrous spatial pyramid pooling network (CASPPNet) is proposed for steel surface defect detection. In CASPPNet, chained atrous spatial pyramid pooling is designed to enlarge receptive field and obtain enrich semantic information. An improved global attention feature fusion module is introduced to achieve feature interaction and salience. Moreover, residual boundary refinement block is introduced to get more complete defect boundary. Comparative experimental results verify that our method is superior to the state-of-the-art segmentation methods on public accessible SD-saliency-900 datasets and can meet the requirement of real-time online detection (the detection efficiency is at over 47 FPS on a single GPU).

Modeling and Calibration of Active Thermal-Infrared Visual System for Industrial HMI

In the industrial application of the human-machine interface (HMI), thermal-infrared cameras can detect objects that are limited by visible-spectrum cameras. It can receive the energy radiated by a target through an infrared detector and obtain the thermal image corresponding to the heat distribution field on the target surface. Because of its special imaging principle, a thermal-infrared camera is not affected by the light source when imaging. Compared to visible-spectrum cameras, thermal imaging cameras can better detect defects with poor contrast but temperature differences or internal defects in products. Therefore, it can be used in many specific industrial inspection applications. However, thermal-infrared imaging has the phenomenon of thermal diffusion, which leads to noisy thermal infrared images and limits its applications in high precision industrial environments. In this paper, we proposed a high precision measurement system for industrial HMI based on thermal-infrared vision. The accurate measurement model of the system was established to deal with the problems caused by imaging noise. The experiments conducted suggest that the proposed model and calibration method is valid for the active thermal-infrared visual system and achieves high precision measurements.

Rapid Nondestructive Detection Enabled by an Ultra‐Broadband NIR pc‐LED

AbstractThe rapid development of near‐infrared (NIR) spectroscopic techniques has greatly stimulated the discovery of novel broadband NIR‐emitting phosphors as advanced light sources. Herein, a novel double‐perovskite phosphor La2MgHfO6:Cr3+/Yb3+ that displays ultra‐broadband NIR emissions with a full‐width at half maximum (FWHM) of 333 nm is reported. The remarkable luminescence property stems from the multiple crystallographic sites, relatively weak crystal field, and efficient Cr3‐to‐Yb3+ energy transfer (ET). The site occupation of Cr3+ is elaborately verified by the Rietveld refinement and first‐principles calculation. By controlling the ET process, the internal/external quantum efficiency (IQE/EQE), bandwidth, and thermal stability of NIR emissions are substantially improved. The as‐prepared phosphors are further integrated into a miniaturized NIR light‐emitting diode (LED) package, demonstrating superior performance in rapid nondestructive detection of structural failure in thin electronic cables. The results described here provide a novel pointcut for designing broadband NIR‐emitting phosphors with desired optical properties toward applications in industrial inspection and medical diagnosis.

Medical Image Enhancement Using Recurrent Neural Networks Based Tv Homomorphic Filter

Abstract: Image Enhancement is one of the important requirements in Digital Image Processing which is important in making an image useful for various applications which can be seen in the areas of Digital photography, Medicine, Geographic Information System, Industrial Inspection, Law Enforcement and many more Digital Image Applications. Image Enhancement is used to improve the quality of poor images. X-ray image contains a large amount of information and became important basis in the process of medical diagnosis. The X-ray image has large gray dynamic range but low contrast. This work proposed a new kind of homomorphic filter with ANN which uses total variation model as the transfer function. it has a good balance in both brightness adjustment and details enhancement. And the comparison results were given, the experimental results show that the method can effectively increase the image contrast, highlight the details. Index Terms: Medical image; X-ray image; total variation (TV); image enhancement; image de-noising; ANN

Prompt gamma-ray methods for industrial process evaluation: A simulation study

Abstract Radioisotope applications in industrial process inspection and evaluation using gamma-ray emitters provide otherwise unavailable information. Offering alternative gamma-ray sources can support the technology by complementing sources’ availability and radiation safety. This work proposes to replace gamma-ray from radioisotopes with prompt gamma-ray from the interaction of neutrons with stable isotopes injected into the industrial process or with the structural material of the industrial process equipment. Monte Carlo N-Particle Transport Code (MCNP5) was used to simulate the irradiation of two-phase flow pipes by 252Cf neutron source. Two simulations were run for each pipe, with and without mixing the liquid phase with the stable isotope 157Gd. The detected gamma-ray spectra were analysed, and images of the two phases inside the pipes were produced. The images were compared to images obtained from simulations of gamma transmission measurement using 60Co. Furthermore, results for prompt gamma computed tomography (CT) were presented and discussed. The studies’ outcomes indicate the potential of prompt gamma-ray to carry out the sealed sources applications of gamma transmission measurements and imaging.

Flying STAR2, a Hybrid Flying Driving Robot With a Clutch Mechanism and Energy Optimization Algorithm

This paper presents FSTAR2, an improved FSTAR design, and a near optimal energy-based algorithm for a flying-driving robot that can navigate in environments crowded with obstacles. The robot can be used for tasks such as package deliveries, industrial inspection, and search and rescue applications. FSTAR2 is fitted with a clutch mechanism that separates the propellers from the motors when driving. Experiments show that the clutch reduces the driving energy consumption by up to 52% and enables driving at higher speeds. To optimize its energy consumption (reducing its normalized energy cost) the FSTAR2 exploits its flying and driving capabilities by implementing a new approach which encloses the search space of the weighted A* algorithm and finds a new weight for the heuristic function, yielding a near optimal solution. Given that both flying and driving speeds are subject to environmental restrictions, the algorithm favors driving whenever possible (driving is almost three times more energy efficient at the considered speed – 2 m/s). Multiple simulations in different environments were conducted and the findings were compared to Dijkstra’s algorithm to estimate the quality of the results. In general, the weighted A* algorithm reduced the number of iterations vs the A* algorithm and produced very similar values to Dijkstra’s Algorithm. The experimental robot can fly like a regular drone, penetrate (25 cm/10 inches diameter) pipes and drive at speeds of up to 4.5 m/s, which is an increase of 50% compared to FSTAR.